A demo agent has one job: look impressive for ninety seconds. A production agent has to be right on the case nobody demoed, cheap enough to run ten thousand times a week, and safe when the model is confidently wrong.
Everything below comes from builds I shipped and then had to keep alive: AgentFlow, a visual no-code builder where non-engineers chain input sources, AI processors and output actions on a canvas, and AgenticAI, a GPT-4o-mini CV screening platform where a wrong answer means a real candidate gets mishandled. Different domains, the same nine lessons.
What does it take to run AI agents in production?
Running AI agents in production takes four things a prototype does not have: measurable output quality, bounded cost, bounded failure, and a way to reconstruct what happened. Every lesson here is a variation on one of those four.
The nine lessons, in the order they usually bite:
- 1Write the eval harness before you write the prompt.
- 2Version prompts, tools and model IDs like code, because they are code.
- 3Give every run a hard token, step and wall-clock budget.
- 4Route cheap steps to cheap models instead of buying one big model for everything.
- 5Retry with jitter and a circuit breaker, never a tight loop.
- 6Put a timeout on every single tool call, shorter than the run budget.
- 7Gate irreversible actions behind a human, not behind a confidence score.
- 8Trace every step: inputs, tool calls, tokens, latency, outcome.
- 9Design the blast radius before you design the capability.
The failure modes I see most often, and the control that actually stops each one:
| Failure mode | What it looks like | Control that fixes it |
|---|---|---|
| Silent quality drift | Outputs still look plausible, users quietly stop trusting them | Scored eval set run on every prompt or model change |
| Cost blowup | Bill jumps with no change in traffic | Hard token, step and wall-clock budget per run |
| Retry storm | Latency spikes and rate-limit errors increase the harder you retry | Exponential backoff with jitter, plus a circuit breaker |
| Hung tool call | Runs never finish, workers saturate, queue backs up | Per-tool timeout shorter than the run budget |
| Irreversible mistake | The agent sends, refunds or deletes the wrong thing | Human approval gate on write actions |
| Unexplainable output | Nobody can say why the agent did that | Per-step trace with inputs, tool calls and token counts |
Lesson 1: Build the eval harness before the prompt
An eval harness is a fixed set of real inputs with known-good outcomes and a scoring function. Build it first, because without it every prompt change is a vibe and every model upgrade is a coin flip.
On the CV screening build the harness came before any prompt tuning: a frozen set of anonymised CVs paired with the ranking a human reviewer had already agreed was correct. Scoring was a mix of exact-match checks on extracted fields and a rubric grade on the reasoning. That harness is what let me swap models and prompt structures without arguing about whether output felt better.
A usable first harness needs less than you think:
- 30 to 100 cases sampled from real traffic, not synthetic examples you invented.
- Deliberate inclusion of the ugly cases: empty fields, wrong language, adversarial input, duplicates.
- Deterministic checks wherever possible (did it extract the right ID?) before you reach for LLM-as-judge.
- A single number per run so regressions are obvious in a diff.
- The harness in CI, so a prompt edit that drops the score fails the same way a broken test does.
"If you cannot score your agent, you are not iterating on it. You are redecorating it."
Why do AI agents blow up your cloud bill?
Agent cost is not a per-call number, it is a multiplication: tokens per step, times steps per run, times retries, times runs per day. A prototype hides that because you personally trigger every run. Production does not hide it.
Where agent spend actually comes from and which lever moves it:
| Cost driver | How it scales | Lever that moves it |
|---|---|---|
| Tokens per step | Context length on every call | Trim retrieved context instead of stuffing whole documents |
| Steps per run | Grows with tool count and task ambiguity | Hard step cap plus a narrower tool list |
| Model tier | Flat multiplier on every call | Route easy steps to a small model, hard steps to a frontier model |
| Retries | Multiplies everything above it | Cap attempts and never retry deterministic failures |
| Run volume | Linear in runs per day | Cache and deduplicate repeated inputs |
Both OpenAI and Anthropic price per input and output token with different rates per model tier, and both revise those rates regularly, so check the current OpenAI and Anthropic Claude pricing pages rather than trusting a number in any blog post, including this one. What does not change is the structure: your bill is tokens multiplied by calls, and the biggest wins come from cutting calls, not from haggling over rates.
The single most effective control I have deployed is a per-run budget object carried through the whole execution: max tokens, max steps, max wall-clock seconds. When any limit trips, the run stops and returns a partial result with a reason. Loops that would have burned a weekend of budget die in eleven steps instead.
What causes retry storms and hung tool calls?
Retry storms happen when every failing request retries immediately and in lockstep, so a provider hiccup becomes a synchronised flood that keeps the provider unhappy. Hung tool calls happen when an HTTP client has no timeout and one slow dependency parks your workers indefinitely.
OpenAI's rate limit documentation is explicit about the fix: back off exponentially and add random jitter so retries from different callers do not land at the same moment. That advice is not provider-specific, it is standard distributed-systems hygiene, and agents violate it constantly because the retry logic gets written at 2am when a demo is failing.
The retry rules I now apply by default
Four rules that have prevented more incidents than any prompt change:
- Retry only transient classes: rate limits, timeouts, 5xx. A schema validation failure will fail identically three more times.
- Exponential backoff with random jitter, capped at three or four attempts.
- A circuit breaker per provider, so when failures cross a threshold you stop calling and serve a degraded path.
- Timeouts on every tool call, set below the run budget, so a hung call cannot outlive the run that owns it.
This matters more for agents than for ordinary API clients because a single agent run fans out into many calls. Ten steps with four retries each is forty chances for one flaky vendor to become your outage. In AgentFlow, where one canvas run can chain model calls with third-party fetch and publish actions, per-tool timeouts were the difference between a slow node and a stuck flow.
Where should a human sit in the loop?
Put the human in front of anything irreversible or externally visible: money moving, messages sent to customers, records deleted, decisions that affect a person. Everything reversible and internal can run unattended.
The useful test is not how confident the model is. Model confidence is not calibrated to your business risk, and a well-phrased wrong answer scores high. The test is what happens if this action is wrong and how expensive the undo is. On the CV screening system the agent ranks, extracts and explains, and a human owns the reject decision, because a wrong rejection is invisible to the system and permanent to the candidate.
A gate matrix I reuse across builds:
| Action type | Reversible? | Default gate |
|---|---|---|
| Read, search, summarise | Yes | Fully autonomous |
| Draft content or internal record | Yes | Autonomous, sampled review |
| Outbound message to a customer | No | Approve first, or approve a template then automate |
| Payment, refund, contract change | No | Explicit human approval, always |
| Delete or overwrite production data | No | Human approval plus soft delete |
Autonomy is a dial, not a switch, and the honest default is to start conservative and loosen it once the eval scores and the incident log earn it. I go deeper on choosing that level in autonomous AI agents explained.
What observability do AI agents in production need?
At minimum you need a trace per run containing every step: the resolved prompt, the tool called, the arguments, the result, token counts, latency and the terminal outcome. Aggregate logs are useless here, because the question is always why did this specific run do that.
The signals worth alerting on:
- Steps per run, p50 and p95. A rising tail means the agent is flailing before it is failing.
- Tokens per run, broken down by input and output, tracked as a cost per successful outcome.
- Tool error rate per tool, not aggregated, so one broken integration is visible immediately.
- Gate rejection rate: how often humans overrule the agent is your live quality metric.
- Eval score on a scheduled replay of the frozen case set, so provider-side model drift surfaces before users report it.
You do not need an exotic platform for this. A traces table in Postgres or Supabase with a JSON payload per step, wired from a FastAPI service, covers most teams for a long time. If you want a vendor-neutral schema instead of inventing one, the OpenTelemetry project maintains GenAI semantic conventions covering spans and attributes for model and agent calls.
Lesson 9: Design the blast radius before the capability
Decide what the worst possible run can touch before you decide what the best run can do. Scoped credentials, allow-listed domains, rate limits per tenant and dry-run modes are cheaper to add on day one than after an incident.
Concretely: give the agent its own service account with the narrowest permissions that work, not a shared admin key. Cap per-tenant spend so one customer's runaway workflow cannot consume the shared budget, which matters enormously in multi-tenant SaaS products. Make destructive tools soft-delete by default. Ship a dry-run flag so you can replay real traffic against a new prompt without side effects.
This is also the honest answer to whether you should build at all. If the blast radius is large and your team cannot staff the on-call, an off-the-shelf tool with narrower scope is the better trade, which I break down in custom AI builds versus AI SaaS tools.
How do you know an agent is ready for production?
It is ready when you can answer six questions without hedging. If any answer is a shrug, you have found your next sprint.
The pre-launch checklist I run before any agent takes real traffic:
- 1What is the eval score on the frozen case set, and what score blocks a deploy?
- 2What is the maximum a single run can cost in tokens, steps and seconds?
- 3What happens when the model provider returns errors for ten minutes straight?
- 4Which actions are gated behind a human, and who is that human at 3am?
- 5Can I reconstruct any individual run from the trace, end to end?
- 6What is the rollback: previous prompt version, previous model ID, or kill switch to the non-agent path?
None of this is exotic. It is the same discipline any distributed system earns, applied to a component that is non-deterministic and metered by the token. If you are earlier in the journey, start with how to build an AI agent from prototype to production, then come back to this list before launch day. When you want the harness, gates and tracing built alongside the agent rather than bolted on afterwards, that is the shape of the AI engineering work I take on.
Key takeaways
- The eval harness is the prerequisite for every other improvement, so build it before tuning prompts.
- Agent cost is tokens multiplied by steps multiplied by retries multiplied by volume, and step count is the cheapest lever to pull.
- Exponential backoff with jitter and a per-provider circuit breaker prevent a vendor hiccup from becoming your outage.
- Every tool call needs a timeout shorter than the run budget that owns it.
- Human gates belong on irreversible actions, decided by undo cost rather than by model confidence.
- A per-step trace with tokens, latency and outcome is the difference between debugging an agent and guessing at it.
Frequently asked questions
- How much does it cost to run AI agents in production?
- Cost equals tokens per step, times steps per run, times retries, times runs per day. Model tier sets the multiplier. Because OpenAI and Anthropic revise per-token rates regularly, check their official pricing pages for current figures, then model your own step count. Cutting steps and trimming context usually saves more than switching vendors.
- What is the difference between an AI agent prototype and a production agent?
- A prototype proves the task is possible on inputs you chose. A production agent handles inputs you did not choose, under a cost ceiling, with retries, timeouts, human gates, tracing and a rollback path. The model work is often identical; the operational scaffolding around it is what takes the extra weeks.
- Can I run AI agents in production without human review?
- Yes, for reversible and internal actions such as reading, searching, summarising or drafting. No, for irreversible ones: payments, deletions, outbound customer messages and decisions about people. Decide by undo cost, not model confidence, because a confidently wrong answer scores just as high as a correct one.
- How many eval cases do I need before shipping an AI agent?
- Start with 30 to 100 real cases sampled from actual traffic, weighted toward the ugly ones: empty fields, wrong language, duplicates and adversarial input. That is enough to catch regressions from prompt and model changes. Grow the set from production failures, adding every incident as a permanent case.
- What causes an AI agent retry storm?
- Immediate, unjittered retries across many concurrent runs. One provider slowdown triggers synchronised retries that keep the provider saturated, so latency and error rates rise together. OpenAI's rate limit documentation recommends exponential backoff with random jitter; adding a per-provider circuit breaker stops the loop from restarting itself.
- Is it worth building a custom AI agent instead of buying an AI SaaS tool?
- It is worth it when the workflow is core to your business, touches your own data, and needs behaviour no vendor exposes. Buy when the scope is narrow and standard. The deciding factor is usually operational capacity: a custom agent means you own evals, observability and on-call forever.
- Which observability metrics matter most for AI agents?
- Steps per run at p50 and p95, tokens per run split by input and output, error rate per individual tool, human gate rejection rate, and a scheduled eval replay score. Together they catch flailing, cost drift, broken integrations, quality problems and provider-side model changes before users report them.
Sources
Written by Syed Husnain Haider Bukhari
AI engineer, data scientist, and founder of Revolutionary Technologies LLC. Ships production AI agents, automations, and data platforms for teams in the US, UK, and UAE — including AgentFlow, AI Walay, and ProLeads.
Get in touch →Related pages
Want this built instead of researched?
Book a 30-minute scoping call. You get a one-page plan and a fixed-scope quote within 48 hours.
Start a conversation