AI Agents in Production: What Actually Works After 6 Months

The Number One Killer: Cost Spirals#

A single unoptimized agent session can burn $10 to $100 in API calls. That is not a typo. Agents make 3 to 10x more LLM calls than a chatbot. One user request triggers planning, tool selection, execution, verification, and response generation. Each step eats tokens.

Three changes cut my spend by 72%:

• Model routing. Route simple tasks to small models. A classification call routed to GPT-4o costs 190x more than the same call on Gemma 4 26B running locally. I tag every agent step with a complexity score and route accordingly.
• Prompt caching. Most agent loops repeat the same system prompt and tool definitions on every call. Provider-native prompt caching (Anthropic, OpenAI, and Google all support it now) cuts repeated context costs by 80-90%.
• Hard budget ceilings. Every agent session gets a token budget. When it hits 80%, the agent switches to a cheaper model. At 100%, it stops and returns a partial result. No exceptions.

const BUDGET_CEILING = 50_000; // tokens per session
const DOWNGRADE_THRESHOLD = 0.8;

async function routeModel(step: AgentStep, usage: TokenUsage) {
  const ratio = usage.total / BUDGET_CEILING;
  if (ratio > DOWNGRADE_THRESHOLD) return 'gemma-4-26b';
  if (step.complexity === 'low') return 'gemma-4-26b';
  return 'claude-sonnet-4';
}

Without these guardrails, one bad loop on a Friday night cost me $340 before I noticed. The budget ceiling would have stopped it at $12.

Observability Is Not Optional#

You cannot debug an agent with console.log. Agents are non-deterministic. The same input produces different execution paths, different tool calls, different token counts. Traditional APM tools miss all of this.

94% of teams with agents in production use dedicated agent observability. The other 6% are either lying or about to have a bad week.

What you need to trace:

• Every LLM call with full prompt, response, latency, and token count
• Tool calls with inputs, outputs, and errors
• The decision chain: why the agent chose tool A over tool B
• Session-level cost accumulation
• Semantic similarity between consecutive outputs (for loop detection)

I settled on Langfuse for tracing because it is open source and self-hostable. Braintrust and LangSmith are solid alternatives if you want managed. The specific tool matters less than having any visibility at all.

import { observeAgent } from '@/lib/agent/observe';

const result = await observeAgent('research-agent', async (trace) => {
  trace.event('plan', { query, modelSelected });
  const plan = await planner.execute(query);

  trace.event('tool-selection', { tools: plan.tools });
  for (const tool of plan.tools) {
    const output = await trace.span(`tool:${tool.name}`, () =>
      tool.execute(plan.context)
    );
    trace.event('tool-result', {
      tool: tool.name,
      tokens: output.usage.total,
      latency: output.durationMs,
    });
  }
  return trace.end(plan.result);
});

Loop Detection Saves You Money and Reputation#

Agents loop. It is their most common failure mode and their most expensive one. A stuck agent will happily burn tokens for hours, generating slight variations of the same wrong answer.

The fix is semantic hashing. After every agent output, compute an embedding vector. If three consecutive outputs have cosine similarity above 0.95, kill the session.

const SIMILARITY_THRESHOLD = 0.95;
const MAX_SIMILAR_STREAK = 3;

function detectLoop(history: EmbeddingVector[]): boolean {
  if (history.length < MAX_SIMILAR_STREAK) return false;
  const recent = history.slice(-MAX_SIMILAR_STREAK);
  return recent.every((vec, i) =>
    i === 0 || cosineSimilarity(vec, recent[i - 1]) > SIMILARITY_THRESHOLD
  );
}

This caught 23 stuck sessions in my first month of deployment. Without it, those sessions would have run until the API rate limit stopped them.

Multi-Agent Orchestration: Keep It Simple#

The orchestrator-worker pattern is the only multi-agent pattern I trust in production. One central agent decomposes tasks and routes to specialists. That is it.

I tried peer-to-peer agent networks. Twice. Both times ended with agents in feedback loops, generating false consensus, and producing confident wrong answers. The GitHub engineering blog documents the same failure pattern.

The Human-in-the-Loop Tax Is Worth Paying#

The industry has landed on a pattern: human-prompted, agent-executed, human-reviewed. Every serious production deployment I know follows this. Full autonomy sounds great in demos. In production, it means an agent sends a customer the wrong refund amount at 3 AM.

The practical version:

1. Human writes a spec or design doc
2. Agent executes scoped tasks against that spec
3. Human reviews every output before it reaches users
4. Agent gets feedback and adjusts

This is slower than full autonomy. But teams that wire in human review ship far more of their agents to production, a pattern LangChain's 2026 State of Agent Engineering report backs up. I will take the tradeoff.

What I Would Do Differently#

If I were starting an agent project today, I would change three things.

Start with one agent, not three. My first deployment used a planner, executor, and reviewer. The planner and reviewer added latency and cost without meaningfully improving output quality. A single agent with good prompts and tool access would have shipped two months earlier.

Instrument from day one. I added observability in month three after a cost incident. By then I had two months of untracked spend and no way to retroactively debug early failures. Tracing should go in before the first agent call.

Set cost alerts at $5/day, not $50/day. My first budget alert was set too high. By the time it fired, the damage was done. Start with an aggressive ceiling and relax it as you understand your baseline.

The teams shipping agents successfully in 2026 are not the ones with the most sophisticated architectures. They are the ones with the best guardrails. Model routing, observability, loop detection, and hard cost ceilings. That is the stack that actually works.

Sources: LangChain's 2026 State of Agent Engineering report, GitHub's engineering blog on multi-agent failures, Braintrust's AI observability guide, and Zylos's agent cost research