AI Agent Tool Chaining Patterns: A Developer's Guide

When an AI agent calls a single tool, it sends a request, gets a result, and moves on. Tool chaining happens when that result feeds into the next tool call, and that result feeds the next, building toward an outcome no single tool could produce.

A practical example: an agent receives a contract PDF, calls a document parser to extract key terms, passes those terms to a search tool to find related agreements, then calls a comparison tool to flag differences. Three tools, each dependent on the previous output.

Production agents average 4 to 8 tool calls per routine task. Complex workflows push higher. The MCPMark benchmark measured an average of 17.4 tool calls per task across realistic scenarios involving databases, file systems, and web automation. The pattern you choose for connecting those calls determines your agent's speed, reliability, and cost.

Six patterns have emerged as the standard vocabulary for tool chaining. Each fits different task shapes. Picking the wrong one means either wasted latency (running things sequentially when they could parallelize) or unreliable results (parallelizing tasks that actually depend on each other).

Anthropic's research on building effective agents, along with implementations across LangGraph, CrewAI, and Google ADK, has converged on six patterns. Here is each one with its mechanics, strengths, and where it breaks down.

1. Sequential Chains

Each tool call waits for the previous one to finish. Output flows forward in a straight line.

Tool A → result → Tool B → result → Tool C → final output

Sequential chains are the simplest to build and debug. Each step can include a gate check, a validation that must pass before the chain continues. If step two produces bad data, you catch it before step three runs.

Best for: Document processing pipelines, multi-stage transformations, any workflow where step N genuinely needs the output of step N-1.

Weakness: Latency scales linearly with the number of tools. Five tools that each take 200ms means a full second of wall-clock time, even if some of those tools could have run simultaneously.

2. Parallel Execution Independent tool calls fire at the same time. Results merge when all calls complete.

┌→ Tool A → result ─┐
├→ Tool B → result ──┤→ merge → final output
└→ Tool C → result ─┘

Parallel execution reduces latency to the duration of the slowest individual call instead of the sum of all calls. Google ADK benchmarks show concrete gains: sequential calls completing in 475ms dropped to 165ms when parallelized, roughly a 2.9x speedup. The LLMCompiler project measured 1.4x to 3.7x latency reductions depending on task structure.

Best for: Gathering information from independent sources, running the same analysis with different parameters, any situation where tools do not depend on each other's output.

Weakness: Merging results adds complexity. If Tool B fails while Tool A and C succeed, you need a strategy: retry Tool B, proceed without it, or abort everything.

3. Conditional Routing

A classifier examines the input and dispatches it to a specialized downstream chain. The agent does not run every tool. It picks the right path.

Input → Router → Path A (structured data tools)
                → Path B (document analysis tools)
                → Path C (web search tools)

LangGraph models this as conditional edges in a directed graph. The router can be an LLM call (classify this input) or a deterministic rule (if file extension is .csv, route to data pipeline).

Best for: Multi-domain systems handling heterogeneous inputs. A customer service agent that routes billing questions to one toolset and technical issues to another.

Weakness: Router accuracy becomes a single point of failure. A misclassified input goes down the wrong path entirely.

4. Orchestrator-Workers

A central agent dynamically decomposes a task into subtasks, delegates each to a worker, and synthesizes the results. Unlike sequential or parallel patterns, the subtasks are not predefined. The orchestrator decides at runtime what needs to happen.

Orchestrator → analyze task → spawn workers
     ├→ Worker 1 (subtask A) → result
     ├→ Worker 2 (subtask B) → result
     └→ Worker 3 (subtask C) → result
Orchestrator → synthesize results → final output

Best for: Open-ended research, code generation across multiple files, any task where you cannot predict the exact steps in advance.

Weakness: The orchestrator itself consumes tokens to plan. Bad decomposition cascades: if the orchestrator misunderstands the task, every worker produces irrelevant output. Debugging is harder because the execution path varies between runs.

5. Evaluator-Optimizer Loop

One agent generates output. A second agent evaluates it against defined criteria and provides feedback. The generator revises. This loops until the evaluator approves or a maximum iteration count is reached.

Generator → draft → Evaluator → feedback ─┐
     ↑                                      │
     └──────────────────────────────────────┘

Best for: Tasks with measurable quality criteria. Code generation (does it pass tests?), translation (does it preserve meaning?), report writing (does it address all requirements?).

Weakness: Each iteration costs a full LLM round-trip. Three revision cycles on a complex prompt can easily consume 10x the tokens of a single-pass approach. Set hard iteration limits.

6. Recursive Supervisor-Subagent

A supervisor agent manages specialized subagents, each of which may call tools or spawn further agents. The supervisor receives results, decides next steps, and may re-delegate. This is the most autonomous pattern and the hardest to control.

Databricks reports that supervisor-subagent architectures now account for 37% of enterprise agent deployments on their platform, making it the dominant pattern for complex business workflows.

Best for: Multi-domain enterprise automation where different subagents handle different knowledge domains (one for structured data, one for unstructured documents, one for external APIs).

Weakness: Debugging is difficult because the execution tree can be deep and unpredictable. Cost scales with depth. Databricks recommends immutable state snapshots at each node so you can replay and diagnose failures.

Choosing a pattern comes down to three factors: does each step depend on the previous output, how much latency can you tolerate, and how predictable does the execution need to be?

Decision Framework

Start sequential. If your task has clear step-by-step dependencies, a sequential chain is the right choice. It is the easiest to build, test, and debug. Anthropic's guidance is explicit: start with the simplest pattern that solves the problem.

Parallelize independent steps. If you find steps in your sequential chain that do not depend on each other, split them into parallel branches. This is the highest-impact optimization for most production agents. Relace's Fast Agentic Search demonstrated a 4x latency reduction by training their model to fire parallel tool calls instead of sequential ones.

Add routing when inputs vary. If your agent handles different input types that need different tool combinations, add a routing layer. Keep the router simple. A misroute is worse than a slow correct path.

Graduate to orchestrator-workers for open-ended tasks. If you genuinely cannot predict the steps in advance, use an orchestrator. But recognize the cost: more tokens, less predictability, harder debugging.

Use evaluator loops for quality-critical output. When the output has objective quality criteria (tests pass, requirements met, compliance checks satisfied), an evaluator loop catches errors that single-pass generation misses.

Reserve recursive patterns for genuine complexity. Multi-agent supervisor architectures are powerful but expensive. Multi-agent workflow adoption on Databricks grew 327% between June and October 2025, but most of that growth came from teams that genuinely needed cross-domain orchestration, not teams adding complexity for its own sake.

Latency Comparison

Sequential execution latency equals the sum of all tool call durations. Parallel execution latency equals the duration of the slowest single call. In practice, benchmarks show parallel execution delivering 1.4x to 3.7x speedups on typical workloads, with some implementations achieving 4x or better through optimized batching.

The tradeoff is that parallel patterns consume the same total compute (or more, due to merging overhead) but return results faster. If your agent runs 8 tool calls and 5 of them are independent, parallelizing those 5 can cut wall-clock time nearly in half.

Give Your Agent Chains Persistent Storage

Fast.io workspaces store intermediate results, index files for semantic search, and hand off output to humans. 50GB free, no credit card, works with any agent framework through the MCP server. Built for agent tool chaining patterns workflows.

Sign Up Free

Tool chaining creates a practical problem: where do intermediate results live between calls? An agent that extracts data in step one, transforms it in step two, and stores it in step three needs somewhere to hold that data reliably across each transition.

The State Problem

Most agent frameworks manage state in memory during a single execution. That works for short chains. It breaks when chains are long-running, when the agent process crashes mid-chain, or when multiple agents need to access shared intermediate results.

Three approaches handle this:

In-memory state works for short, fast chains where a process crash means you restart. LangGraph provides built-in state checkpointing that can persist to disk, giving you replay capability without external infrastructure.

Database-backed state suits production systems where chains may take minutes or hours. Databricks recommends immutable state snapshots: each agent step reads a versioned state object and produces a new version, preventing corruption when async agents overlap.

File-based persistence through a shared workspace gives you both durability and visibility. Intermediate results are stored as files that humans can inspect, agents can access, and audit trails can track.

Why Shared Workspaces Fit Tool Chains

When agents chain tools across steps that produce files (extracted documents, generated reports, transformed datasets), those files need a home that is not ephemeral memory and not a raw object store that only the agent can see.

Local filesystems work during development but fail in production where agents run on ephemeral compute. S3 and similar object stores provide durability but lack the collaboration layer: no previews, no search, no way for a human to review intermediate output without downloading files and opening them locally.

Fast.io workspaces address this by giving agents and humans the same persistent layer. An agent chains tools that produce files, stores intermediate results in a workspace, and the workspace automatically indexes those files for semantic search and AI chat through Intelligence Mode. The human reviewer does not need to parse raw S3 paths. They open the workspace, see the files, and can ask Ripley questions about the content.

The Fast.io MCP server exposes workspace operations, storage, AI queries, and workflow tools through a single Streamable HTTP endpoint. An agent running a tool chain can upload intermediate results, query previous outputs with semantic search, and hand off the final result to a human, all through the same interface. File locks prevent conflicts when multiple agents access shared state.

The free agent plan includes 50GB storage, 5,000 monthly credits, and 5 workspaces with no credit card required, which covers most development and early production workloads.

Each major agent framework implements tool chaining differently. Understanding the framework's model helps you pick the right one for your chain topology.

LangGraph

LangGraph models chains as directed graphs where nodes are processing steps and edges define transitions. Conditional edges enable routing patterns. Cycles enable evaluator-optimizer loops. Built-in state checkpointing supports replay and time-travel debugging.

LangGraph gives you the most control over execution flow but has the steepest learning curve. If your chain topology is complex (conditional branches, loops, parallel fan-out with merge), LangGraph handles it natively.

CrewAI

CrewAI uses role-based agents that collaborate through structured process types: sequential pipelines and hierarchical orchestration. You define agents with roles, goals, and backstories, then compose them into "crews" that execute tasks in order.

The framework benchmarks at 82% task success rate with 1.8-second average response latency. It reads like English, which makes it the fast path to a working multi-agent chain. The tradeoff is less flexibility in custom routing and conditional logic compared to LangGraph.

Google ADK

Google's Agent Development Kit provides explicit SequentialAgent and ParallelAgent primitives. If you need parallel tool calling with measured latency gains and tight Gemini integration, ADK handles this with minimal boilerplate.

AutoGen

Microsoft's AutoGen (now AG2) models agent interaction as multi-turn conversations managed by a GroupChatManager. Agents debate and converge through dialogue rather than structured graph execution. This is ideal for research workflows and consensus-building but less predictable for production pipelines.

Direct API Calls

Anthropic recommends starting without a framework at all. Direct API calls to Claude, GPT-4, or other models give you full visibility into what is happening at each step. You control the prompt, the tool definitions, and the chain logic explicitly. Add a framework only when the complexity justifies it.

Connecting Framework Chains to Persistent Storage

Regardless of framework, the persistence question remains. Framework-managed state handles in-process coordination. For durable storage, audit trails, and human handoff, agents can write intermediate and final outputs to a workspace. The Fast.io MCP server works with any framework since it exposes standard MCP tooling over Streamable HTTP at /mcp. An agent running in LangGraph, CrewAI, or raw API calls can upload results, create tasks for human review, and transfer workspace ownership when the chain completes.

Moving tool chains from prototypes to production surfaces problems that do not appear in demos.

Error Handling and Retry Strategy

Every tool in a chain can fail. Network timeouts, rate limits, malformed responses, and upstream service outages all interrupt execution. Production chains need a strategy for each failure mode.

For sequential chains, a failure at step N means steps N+1 onward cannot execute. Options: retry with exponential backoff, substitute a fallback tool, or abort and surface the error. The right choice depends on whether the chain is idempotent. If step N already wrote to a database, retrying from the beginning may create duplicates.

For parallel chains, partial failure is the harder problem. If three tools run simultaneously and one fails, do you use the two successful results and note the gap? Retry just the failed tool? Or discard everything and retry the full batch? Most production systems use a "proceed with partial results plus retry" approach, logging the failure for later review.

Cost Management

Each tool call in a chain typically involves an LLM round-trip to decide the next action, plus the tool execution itself. Longer chains consume more tokens. Evaluator-optimizer loops multiply this by the number of iterations.

Practical strategies: set hard limits on chain length (maximum 12 tool calls before forced completion), use cheaper models for routing decisions, and cache tool results that are likely to be reused.

Observability

When a chain produces unexpected output, you need to trace which tool call went wrong. Log every tool call with its input, output, duration, and the LLM reasoning that triggered it. Databricks recommends immutable state snapshots at each step so you can replay the exact execution path.

Agent activity stored in a shared workspace creates a natural audit trail. Each file upload, task creation, and workflow step is logged with timestamps, giving both the agent developer and the human reviewer visibility into what happened and when. Fast.io's audit trails capture these events automatically when agents operate through the MCP server or API.

When to Keep It Simple

The strongest signal from practitioners building production agents: most teams over-architect their tool chains. A sequential chain of 3 to 5 tool calls with good error handling covers the majority of real-world agent tasks. Reach for parallel execution when you have genuinely independent steps and measurable latency requirements. Reach for multi-agent orchestration only when different steps require fundamentally different capabilities or knowledge domains.

The Databricks documentation puts it well: deterministic chains are "often the sweet spot for enterprise use cases, simpler to debug than multi-agent setups while still allowing dynamic logic."