Conditional Parallel Tool Execution: Agent Pattern

I run tool calls together only when they have permission, valid inputs and no dependencies or shared-state conflicts. If one call needs another’s result, I keep them in order.

For example, I can check an order and fetch delivery guidance together if both already have their inputs. A refund needs its own authorisation and sequence.

My approach follows 3 steps:

1. Select: Check the agent’s state, permissions, arguments and dependencies. Record which calls can start.
2. Run: Limit concurrent calls, queue the rest and set timeouts. Before retrying, check whether the action finished; reuse its idempotency key when a retry is safe.
3. Merge: Match results to call IDs, keep successful results, flag gaps or conflicts and update the state once. Track queued and incomplete calls until completion or deadline.

I test failure paths before enabling autonomous execution. That includes rejected calls, queue limits, cancellation and uncertain outcomes that need human review. I then compare latency, reliability and £ per completed task with serial execution, using the same workload and keeping sensitive data out of logs.

::: @figure {Conditional Parallel Tool Execution: Select, Run, Merge} :::

#11 · Parallel Tool Calls – Make Your AI Agent Faster | Agentic AI Course

Step 1: Check Conditions and Select Tool Calls

Use a router to turn the current agent state into a call plan: no calls, one call, or a parallel batch. Apply concurrency limits in the next step.

Check Permissions, Arguments and Dependencies

Confirm that tool use is needed, the caller has permission to use the tool, and the arguments match the schema and current prerequisites. Defer the call if inputs are missing.

Treat user-controlled content as untrusted. Keep agents that read it separate from agents with access to private data or external communications [1].

Check data dependencies before scheduling calls in parallel.

Record the Call Plan and Dependencies

Record the routing decision and dependencies for each planned call. Only independent, eligible calls should run in parallel.

plan = router.select(current_state)   # may contain zero calls
save(plan.ids, plan.dependencies, status="selected")  # Record the selected plan

if plan.is_empty():
    continue_reasoning_without_tools()
else:
    ready = evaluate_conditions_and_dependencies(plan, current_state)
    independent, serial = separate_independent_calls(ready)
    batch = independent if independent else serial.take_one()

    # Queue only the ready batch
    mark_queued(batch.ids)

The next step executes only the ready batch, within concurrency limits.

Step 2: Run Calls Within Concurrency Limits

Take the ready batch from Step 1 and run only eligible, independent calls through a bounded worker pool. Queue any excess calls. Under each call ID, record its status, result and error. Then standardise the returned results before reasoning continues.

Set Limits, Timeouts and Retry Rules

Size the semaphore or worker pool to match the available connections and worker slots. Apply rate limits where needed to stay within the downstream service’s request capacity.

Set a timeout for each call and cap retries, using backoff for temporary failures. Before retrying a call that times out or receives no response, check its external status.

If completion is confirmed, record the receipt and do not repeat the action. If completion is not confirmed, retry only when it is safe, using the same idempotency key. If the call is still processing, wait before checking again. Send any outcome that remains unknown for human review.

Compare Serial and Parallel Execution

Run the same call plan under serial and bounded execution, measuring total elapsed time, resource use and cost impact. Bounded parallelism lowers latency while keeping calls within downstream limits. Serial execution is simpler, but slower.

Step 3: Combine Results and Continue Reasoning

After execution, merge the outcomes against the recorded call plan, including queued and incomplete calls. Track every selected call until it reaches a terminal status or its deadline expires - even queued calls that never started.

Standardise Results and Keep Partial Success

Keep usable results from successful calls, and clearly label any missing data.

Resolve Conflicts and Update Agent State

Use fixed rules to handle duplicate fields and conflicting values. Flag anything those rules cannot resolve.

Validate the merged results, then apply a single state update. Continue only when dependencies are satisfied. Otherwise, request the missing information and leave the item unresolved. Re-check the updated state before the next reasoning turn.

Conclusion: Test and Run the Pattern Safely

Test Routing, Queues and Partial Failures

Test four cases: no eligible calls, one eligible call, several independent calls and mixed dependencies. Check which calls may start together, not just which results appear. Include permission denials and rejected arguments.

Also test over-limit batches, timeouts, cancellation, retries and partial failures. Verify idempotency, confirm that successful results are preserved and check that each result matches its call ID.

Check Readiness for Production

Once routing and failure-path tests pass, use a release checklist to check readiness for rollout. Cover condition logic, dependency checks, run-time authorisation, concurrency and rate limits, result correlation and timeout handling. Every check must pass before you enable autonomous execution.

Monitor routing decisions, queue depth and outcomes, but leave out sensitive data. Compare latency, reliability and cost (£ per completed task) against serial execution, using the same workload mix for both runs.

FAQs