Agent demos work great on Twitter. Agent products fall over in week two of real usage. The difference is almost always architecture — patterns the team adopted (or didn't) that determine whether the system holds up when users do unpredictable things and the model has a bad day.
These are the patterns that survive contact with real users. They're not exotic. They're what experienced teams converge on after a few painful incidents.
Pattern 1: Plan-then-execute
Before any tools run, the agent produces a structured plan. The plan is reviewed — by another model, a validator, or at high stakes a human — and only then does execution begin. Catches bad plans before they touch real systems.
Good for: tasks with consequences (database writes, payments, customer communication, irreversible actions).
How it looks:
- Agent receives task: "refund this customer's last order"
- Agent emits a plan: [find order, verify refund eligibility, issue refund, send confirmation email]
- Validator checks the plan against business rules
- If valid, plan executes step-by-step
The trick: the plan is a structured artifact (JSON, not free text). You can inspect, validate, and reject it before any tool fires.
Pattern 2: Hierarchical routing
A small, fast router classifies the request and delegates to a specialized sub-agent. Each sub-agent has a narrow toolset and a focused prompt. This beats one giant generalist agent on accuracy and cost.
Treat that router as an evaluated policy, not a cheap preliminary call. The model routing and cascading framework measures escalation recall, tail latency, and cost per successful task.
Good for: products with many distinct user intents (support assistants, multi-domain copilots, internal ops bots).
The router uses a cheap model — Haiku, GPT-4o-mini, or similar. Sub-agents use mid-tier or top-tier models. Total system cost is dramatically lower than running everything through GPT-4, and accuracy is usually higher because each sub-agent has a smaller surface area to fail on.
Pattern 3: Verifier loops
After the agent produces a result, a second pass verifies it against the original request. The verifier can be a separate model or the same model with a different prompt. If verification fails, the agent gets another try with the failure as context.
Good for: tasks where wrong-but-plausible answers are common (math, code, structured generation).
Verifiers don't fix everything. Models still hallucinate. But a verifier catches a class of confidently-wrong outputs that no amount of clever prompting will catch.
Pattern 4: Tool selection by retrieval
If you have many tools, don't put them all in every prompt. Embed tool descriptions, retrieve the top-K most relevant for the query, and only expose those.
Two benefits: prompts stay focused (better accuracy on tool calls), and tokens stay down (lower cost, less latency).
Good for: systems with 20+ tools, especially internal ops agents that have access to many APIs.
Pattern 5: Bounded autonomy
Give the agent a budget — max tool calls, max time, max cost per task. Refuse to continue past it. Most catastrophic agent failures involve runaway loops where the agent keeps trying variations of the same failing action.
A hard limit beats every prompt-level guardrail you can write. The model can't promise to stop; the runtime can enforce it.
Common limits:
- Max tool calls per task (often 10-20)
- Max LLM tokens per task (a few hundred thousand)
- Max wall-clock time (often 60-300 seconds)
- Max retries per failing tool call (often 3)
Pattern 6: Human-in-the-loop for high-stakes actions
For any action with consequences — payments, customer messages, data deletion — require human approval before execution. The agent does all the work up to the approval gate, presents what it's about to do, and waits.
This isn't a fallback for failure. It's the design. Many production "agents" are really powerful drafters with approval gates, and that's the right architecture for the next year or two of capability.
What doesn't work as well as people claim
- Pure ReAct for production — the unstructured "think → act → observe" loop drifts in long tasks. Combine with explicit planning.
- Self-correction without a verifier — models often confidently produce the same wrong answer twice. They need an external check.
- Letting agents write to production data unsupervised — always combine plan-then-execute with bounded autonomy, ideally with a human approval step.
- Long memory without compaction — agents that accumulate every observation slow down and lose accuracy. Compact older memory aggressively.
- Multi-agent for the sake of multi-agent — coordinated multi-agent systems are hard to debug. Use one agent with retrieval over good context first, then apply the single-agent versus multi-agent decision framework before adding coordination layers.
The throughline
Reliable agents look more like compilers than like humans. Plan, verify, bound, route. The patterns that work in production are the ones that make agent behavior boring. Boring is good. Boring means you can ship it on Friday and not lose sleep about Monday.
Keep those patterns boring after every prompt, model, tool, and policy change with an AI-agent regression-testing workflow in CI.
If you want to learn these patterns by building three production agents end-to-end, the JoinAI MasterClass walks through each of them in a cohort setting.
Build the next layer
Patterns become reliable only when their boundaries are explicit. Start with tool schemas, permissions, and failure contracts, then decide where a human approval step belongs in the agent loop.
When the behavior still fails, diagnose the earliest broken contract in the agent trace instead of patching the final response. If those tools are exposed through Model Context Protocol, apply the separate MCP authentication and tool-boundary controls.
To practice defending these boundaries under time pressure, work through the AI agent system design interview questions, including the complete support-and-refund design exercise.
