AI FinOps Note 003 · Token Waste
The leftover prompt is the bill.
When I look at an AI invoice, I do not start with the model ID. I start with whether last quarter's instructions are still running extra tool calls this quarter.
On 8 September 2026, the ClaudeDevs account posted an X article, Reducing cost and improving performance with Claude Platform. It is signed by Lance Martin, Brad Abrams, Isabella He, and Ben Lehrburger. Their claim is that many Claude Platform applications can cut cost without giving up performance by doing three things: raise the prompt-cache hit rate, strip anti-patterns left over from older models, and set effort to the task. They put that guidance in a claude-api skill for Claude Code.
That is a FinOps claim, not a model-shopping claim. If a deployment cannot show accepted output and hours saved on the roster, a cheaper list price does not rescue it. Their article is useful because it names waste that shows up as tokens and tool calls rather than as another vendor.
Pay once for the stable prefix.
Before Claude answers, it prefills the prompt into an internal working state. That prefill is the expensive part of handling input. Prompt caching saves the key-value state. When the next request starts with the same prefix, Claude reads it back. Cache reads are billed at a fraction of full input price. The cache is pinned to one model, the prefix must match byte for byte, and it has a time-to-live.
The miss reasons they list are operational, not theoretical. Effort settings render into the prompt ahead of the content, so changing effort mid-conversation breaks the prefix except on a few models including Opus 5 and Fable 5.1. A timestamp or ID in the system prompt changes every call. Tool definitions sit at the top of the messages API; reordering them breaks the cache. A subagent only shares the parent's cache if the fork is byte-identical, on the same model, at the same effort. The five-minute TTL starts at the request. If a parent waits more than five minutes on tools or subagents, its cache is gone before the result returns. They suggest a one-hour TTL on that prefix instead.
The fixes are layout, not heroics. Keep volatile values out of the prefix. Defer rarely used tools so they stay out of the cached block until looked up. Put static tools and the system prompt first, conversation after. Switch model or effort when compaction is already going to miss. Pre-warm with max_tokens 0 and the same effort as live traffic so the first real request hits a warm cache. I treat cache hit rate the way I treat idle VMs: if nobody can say why it missed, the spend is unowned.
Last year's patch is this year's loop.
Prompts accumulate lines that patched an older model's weaknesses. Frontier models take those lines literally. "Verify twice" duplicated order lookup on every refund in their Opus 5 support test. "Be maximally thorough" became dozens of unneeded knowledge-base searches. A mandatory scratchpad stacked on top of native reasoning; on three tickets the model wrote the tool call inside thinking and never executed it. Contradictory refund rules made Opus 5 withhold four refunds it owed. A retired thinking setting made the API reject every routing request.
They planted those anti-patterns one at a time on a clean prompt, six legacy prompts in total, and ran /claude-api prompt-audit once per prompt. Across the six, they report costs down 14.6 percent and accuracy up 5.3 percent on average after the audit. Cost fell because extra tool calls and duplicated reasoning went away. Accuracy rose because the API stopped rejecting work, the model stopped asking for confirmation instead of issuing owed refunds, and the scratchpad stopped eating the tool call.
That is the hours-saved test. Extra searches are roster time even when nobody is watching the loop. I would not copy their 14.6 percent onto another queue. It is their planted-prompt average on one support benchmark. The mechanism is still the one I use: if an instruction was written to restrain a weaker model, check whether it now spends tokens the current model does not need.
Max is not a policy.
Effort is how hard Claude is told to work. Low effort generally finishes faster. High effort deliberates, verifies, and explores alternatives. On FrontierCode Diamond, the hardest 50 tasks, they report Claude Fable 5 at 11.5 percent for USD 5.35 per task at low effort, and 30.9 percent for USD 19.00 at max. Effort raised the score about 2.7 times for about 3.5 times the cost. On Humanity's Last Exam without tools, Fable 5.1 scores about 53 percent at low effort for about USD 0.30 per question and about 61 percent at max for about USD 2.23. The last step to max adds about half a point for 46 percent more cost, inside the benchmark's run-to-run noise.
High effort can over-think. Low effort can answer from the first search instead of the third. Their calibration is to measure the actual task. On CursorBench 3.2, Fable 5.1 at low effort matched Fable 5 at high effort at a third of the cost. Two reasons: less work per task at low effort, and prompt-cache reads priced at USD 0.25 per million tokens on 5.1 versus USD 1.00 on Fable 5. Even at Fable 5's cache prices, they say 5.1 at low effort would still cost about 40 percent less.
They ran /claude-api hillclimb on a customer support benchmark starting from Opus 4.8 at default high effort. The search tried Opus 5 at low effort after prompt-audit, cleared 98.9 percent train accuracy at 2.6 cents per ticket, then stepped to Sonnet 5 at low effort at 1 cent per ticket. Accuracy fell to 88.9 percent, then routing rules and a refund-cap cross-reference brought it back to 98.9 percent at the same 1 cent. On 14 held-out tickets the search never saw, the final setup scored 90.5 percent against 78.6 percent for the original, at about one-fifth the cost.
That is the accepted-output test. Same tickets, fewer cents, higher held-out accuracy. It is still their benchmark and a 14-ticket holdout. I would not treat one-fifth as a planning factor for another workflow. I would treat the sequence as the work: measure the queue, strip leftover rituals, then pick model and effort against that queue, not against a default of max.
Sources
-
ClaudeDevs, Reducing cost and improving performance with Claude Platform, 8 Sep 2026
X article posted by the official Claude developers account. Written by Lance Martin, Brad Abrams, Isabella He, and Ben Lehrburger. Cache rules, anti-pattern counts, effort prices, and the hillclimb figures are theirs. The stills are from that article.