benchrWhen it breaks
The prototype cost pennies and the real thing costs a fortune
Also asked as my API bill exploded · why is this so expensive · token costs out of control · the agent burned through credits
Cause documented by a vendor or a published paper Not tested by benchr First logged Last checked
What is actually happening
Almost always one of three things: the same context is being re-sent and re-charged on every turn, work that could wait is being run at the live rate, or an agent is looping.
Why
- A conversation re-sends its history every turn. Without caching, an unchanged 50-page attachment is paid for again on each message.
- Prompt caching and batch processing are both documented, priced features — the discounts exist and are simply not on by default.
- Agent loops multiply everything: one wrong turn repeated is the whole budget.
The quick fix
Find the largest unchanging block in your prompt. If it is re-sent every turn and not cached, that single change is usually most of the bill.
The real fix
Cache the stable prefix, move anything that is not interactive to the batch rate, and put a hard spend ceiling on any agent that can loop. Then price the route against the published rates rather than guessing.
Step by step
- Measure before optimizing: which requests, which models, which token counts.
- Cache the system prompt and any long stable attachment.
- Move overnight and bulk work to batch pricing.
- Cap agent iterations and add a spend limit that stops the run, not one that emails you afterwards.
- Re-check the published prices — rates change, and an old number in your model is its own cost bug.
If you are seeing an API error
Grounded in
- Stop paying twice for the same fileThe caching and batch discounts are documented and priced by the providers; the figures live in benchr's pricing ledger.