Agent Design Patterns That Actually Work in Production

Agent demos work great on Twitter. Agent products fall over in week two of real usage. The difference is almost always architecture — patterns the team adopted (or didn't) that determine whether the system holds up when users do unpredictable things and the model has a bad day.

These are the patterns that survive contact with real users. They're not exotic. They're what experienced teams converge on after a few painful incidents.

Pattern 1: Plan-then-execute

Before any tools run, the agent produces a structured plan. The plan is reviewed — by another model, a validator, or at high stakes a human — and only then does execution begin. Catches bad plans before they touch real systems.

Good for: tasks with consequences (database writes, payments, customer communication, irreversible actions).

How it looks:

1. Agent receives task: "refund this customer's last order"
2. Agent emits a plan: [find order, verify refund eligibility, issue refund, send confirmation email]
3. Validator checks the plan against business rules
4. If valid, plan executes step-by-step

The trick: the plan is a structured artifact (JSON, not free text). You can inspect, validate, and reject it before any tool fires.

Pattern 2: Hierarchical routing

A small, fast router classifies the request and delegates to a specialized sub-agent. Each sub-agent has a narrow toolset and a focused prompt. This beats one giant generalist agent on accuracy and cost.

Good for: products with many distinct user intents (support assistants, multi-domain copilots, internal ops bots).

The router uses a cheap model — Haiku, GPT-4o-mini, or similar. Sub-agents use mid-tier or top-tier models. Total system cost is dramatically lower than running everything through GPT-4, and accuracy is usually higher because each sub-agent has a smaller surface area to fail on.

Pattern 3: Verifier loops

After the agent produces a result, a second pass verifies it against the original request. The verifier can be a separate model or the same model with a different prompt. If verification fails, the agent gets another try with the failure as context.

Good for: tasks where wrong-but-plausible answers are common (math, code, structured generation).

Verifiers don't fix everything. Models still hallucinate. But a verifier catches a class of confidently-wrong outputs that no amount of clever prompting will catch.

Pattern 4: Tool selection by retrieval

If you have many tools, don't put them all in every prompt. Embed tool descriptions, retrieve the top-K most relevant for the query, and only expose those.

Two benefits: prompts stay focused (better accuracy on tool calls), and tokens stay down (lower cost, less latency).

Good for: systems with 20+ tools, especially internal ops agents that have access to many APIs.

Pattern 5: Bounded autonomy

Give the agent a budget — max tool calls, max time, max cost per task. Refuse to continue past it. Most catastrophic agent failures involve runaway loops where the agent keeps trying variations of the same failing action.

A hard limit beats every prompt-level guardrail you can write. The model can't promise to stop; the runtime can enforce it.

Common limits:

• Max tool calls per task (often 10-20)
• Max LLM tokens per task (a few hundred thousand)
• Max wall-clock time (often 60-300 seconds)
• Max retries per failing tool call (often 3)

Pattern 6: Human-in-the-loop for high-stakes actions

For any action with consequences — payments, customer messages, data deletion — require human approval before execution. The agent does all the work up to the approval gate, presents what it's about to do, and waits.

This isn't a fallback for failure. It's the design. Many production "agents" are really powerful drafters with approval gates, and that's the right architecture for the next year or two of capability.

What doesn't work as well as people claim

• Pure ReAct for production — the unstructured "think → act → observe" loop drifts in long tasks. Combine with explicit planning.
• Self-correction without a verifier — models often confidently produce the same wrong answer twice. They need an external check.
• Letting agents write to production data unsupervised — always combine plan-then-execute with bounded autonomy, ideally with a human approval step.
• Long memory without compaction — agents that accumulate every observation slow down and lose accuracy. Compact older memory aggressively.
• Multi-agent for the sake of multi-agent — coordinated multi-agent systems are hard to debug. Use one agent with retrieval over good context first.

The throughline

Reliable agents look more like compilers than like humans. Plan, verify, bound, route. The patterns that work in production are the ones that make agent behavior boring. Boring is good. Boring means you can ship it on Friday and not lose sleep about Monday.

If you want to learn these patterns by building three production agents end-to-end, the JoinAI MasterClass walks through each of them in a cohort setting.