What we learned shipping AI agents
Agents promise autonomy, but production teaches humility. Lessons on tools, observability, and knowing when not to use an agent.
Agentic systems can plan, call tools, and complete multi-step work on their own. They are also harder to make reliable than a single model call. Here is what has held up for us in production.
Not everything needs an agent
The first question we ask is whether an agent is even the right tool. A fixed workflow is easier to test, cheaper to run, and simpler to reason about. Reach for an agent when the path genuinely varies from run to run.
Tools are the interface to reality
An agent is only as capable as the tools you give it. We design tools like a public API:
- Narrow, well-named, and single-purpose
- Strict input validation on every call
- Clear, structured errors the model can recover from
Observability is non-negotiable
You cannot debug what you cannot see. Every agent run emits a full trace — each step, tool call, and decision — so we can replay failures and measure behavior over time.
Contain the blast radius
// Bound autonomy: cap steps and require approval for risky actions.
const result = await runAgent(task, {
maxSteps: 12,
requireApproval: (action) => action.kind === "write",
});
Step limits, timeouts, and human approval for high-impact actions keep an autonomous system from turning a small mistake into a large one.
The takeaway
Treat agents as software, not magic. The teams that succeed are the ones that bring the same rigor — testing, observability, and guardrails — that they bring to any other production system.