AI FinOps

Notes on the economics and operation of enterprise AI.

Leslie Li

AI FinOps Note 003 · Token Waste

The leftover prompt is the bill.

When I look at an AI invoice, I do not start with the model ID. I start with whether last quarter's instructions are still running extra tool calls this quarter.

Leslie Li 5 min read Observations from IT practice

On 8 September 2026, the ClaudeDevs account posted an X article, Reducing cost and improving performance with Claude Platform. It is signed by Lance Martin, Brad Abrams, Isabella He, and Ben Lehrburger. Their claim is that many Claude Platform applications can cut cost without giving up performance by doing three things: raise the prompt-cache hit rate, strip anti-patterns left over from older models, and set effort to the task. They put that guidance in a claude-api skill for Claude Code.

That is a FinOps claim, not a model-shopping claim. If a deployment cannot show accepted output and hours saved on the roster, a cheaper list price does not rescue it. Their article is useful because it names waste that shows up as tokens and tool calls rather than as another vendor.

Illustration from the ClaudeDevs article: four stacked cards labeled Instructions.
From the 8 Sep 2026 article. Four stacked cards labeled Instructions.
01 · Cache

Pay once for the stable prefix.

Before Claude answers, it prefills the prompt into an internal working state. That prefill is the expensive part of handling input. Prompt caching saves the key-value state. When the next request starts with the same prefix, Claude reads it back. Cache reads are billed at a fraction of full input price. The cache is pinned to one model, the prefix must match byte for byte, and it has a time-to-live.

The miss reasons they list are operational, not theoretical. Effort settings render into the prompt ahead of the content, so changing effort mid-conversation breaks the prefix except on a few models including Opus 5 and Fable 5.1. A timestamp or ID in the system prompt changes every call. Tool definitions sit at the top of the messages API; reordering them breaks the cache. A subagent only shares the parent's cache if the fork is byte-identical, on the same model, at the same effort. The five-minute TTL starts at the request. If a parent waits more than five minutes on tools or subagents, its cache is gone before the result returns. They suggest a one-hour TTL on that prefix instead.

Article figure of a Claude Console panel: cache read ratio 71.3 percent, cache read tokens 12.4 billion, and Opus 5 miss reasons Messages changed, System changed, Tools changed, Model changed.
Figure 1 from the article. Their caption: Claude Console can diagnose unexpected prompt cache misses by comparing requests and identifying exactly where the prompt prefix diverged. On the panel: cache read ratio 71.3% versus the previous 7 days, cache read tokens 12.4B, and 1.2B missed tokens on Opus 5 split across Messages, System, Tools, and Model changed.

The fixes are layout, not heroics. Keep volatile values out of the prefix. Defer rarely used tools so they stay out of the cached block until looked up. Put static tools and the system prompt first, conversation after. Switch model or effort when compaction is already going to miss. Pre-warm with max_tokens 0 and the same effort as live traffic so the first real request hits a warm cache. I treat cache hit rate the way I treat idle VMs: if nobody can say why it missed, the spend is unowned.

02 · Instructions

Last year's patch is this year's loop.

Prompts accumulate lines that patched an older model's weaknesses. Frontier models take those lines literally. "Verify twice" duplicated order lookup on every refund in their Opus 5 support test. "Be maximally thorough" became dozens of unneeded knowledge-base searches. A mandatory scratchpad stacked on top of native reasoning; on three tickets the model wrote the tool call inside thinking and never executed it. Contradictory refund rules made Opus 5 withhold four refunds it owed. A retired thinking setting made the API reject every routing request.

They planted those anti-patterns one at a time on a clean prompt, six legacy prompts in total, and ran /claude-api prompt-audit once per prompt. Across the six, they report costs down 14.6 percent and accuracy up 5.3 percent on average after the audit. Cost fell because extra tool calls and duplicated reasoning went away. Accuracy rose because the API stopped rejecting work, the model stopped asking for confirmation instead of issuing owed refunds, and the scratchpad stopped eating the tool call.

Article scatter chart of cost per ticket versus accuracy for Opus 4.8 legacy, Opus 5 unaudited, and Opus 5 audited, with a /claude-api prompt-audit step marked plus 5.3 points and minus 0.49 cents.
Figure 3 from the article. Their caption: the effect of prompting anti-patterns during model migration from Opus 4.8 to Opus 5. On the chart: Opus 4.8 legacy 2.52¢ at 89.4%, Opus 5 unaudited 3.43¢ at 91.7%, Opus 5 audited 2.93¢ at 97.0%. The audit step is labeled +5.3 pts, -0.49¢.

That is the hours-saved test. Extra searches are roster time even when nobody is watching the loop. I would not copy their 14.6 percent onto another queue. It is their planted-prompt average on one support benchmark. The mechanism is still the one I use: if an instruction was written to restrain a weaker model, check whether it now spends tokens the current model does not need.

03 · Effort

Max is not a policy.

Effort is how hard Claude is told to work. Low effort generally finishes faster. High effort deliberates, verifies, and explores alternatives. On FrontierCode Diamond, the hardest 50 tasks, they report Claude Fable 5 at 11.5 percent for USD 5.35 per task at low effort, and 30.9 percent for USD 19.00 at max. Effort raised the score about 2.7 times for about 3.5 times the cost. On Humanity's Last Exam without tools, Fable 5.1 scores about 53 percent at low effort for about USD 0.30 per question and about 61 percent at max for about USD 2.23. The last step to max adds about half a point for 46 percent more cost, inside the benchmark's run-to-run noise.

High effort can over-think. Low effort can answer from the first search instead of the third. Their calibration is to measure the actual task. On CursorBench 3.2, Fable 5.1 at low effort matched Fable 5 at high effort at a third of the cost. Two reasons: less work per task at low effort, and prompt-cache reads priced at USD 0.25 per million tokens on 5.1 versus USD 1.00 on Fable 5. Even at Fable 5's cache prices, they say 5.1 at low effort would still cost about 40 percent less.

They ran /claude-api hillclimb on a customer support benchmark starting from Opus 4.8 at default high effort. The search tried Opus 5 at low effort after prompt-audit, cleared 98.9 percent train accuracy at 2.6 cents per ticket, then stepped to Sonnet 5 at low effort at 1 cent per ticket. Accuracy fell to 88.9 percent, then routing rules and a refund-cap cross-reference brought it back to 98.9 percent at the same 1 cent. On 14 held-out tickets the search never saw, the final setup scored 90.5 percent against 78.6 percent for the original, at about one-fifth the cost.

Article chart of four hillclimb steps from Opus 4.8 high effort to Sonnet 5 low effort, plotted as decision accuracy on the train split against cost per ticket on a log scale.
Figure 6 from the article. Their caption: hillclimbing improves cost and performance. On the chart, train-split decision accuracy against cost per ticket: Opus 4.8 high effort starts at 74.4%, then Opus 5 low effort, then Sonnet 5 low effort, then Sonnet 5 low effort after improve prompt, ending near 1¢. The 98.9% and 14-ticket holdout numbers in the prose are from the article text, not labels on this frame.

That is the accepted-output test. Same tickets, fewer cents, higher held-out accuracy. It is still their benchmark and a 14-ticket holdout. I would not treat one-fifth as a planning factor for another workflow. I would treat the sequence as the work: measure the queue, strip leftover rituals, then pick model and effort against that queue, not against a default of max.

Sources

  1. ClaudeDevs, Reducing cost and improving performance with Claude Platform, 8 Sep 2026

    X article posted by the official Claude developers account. Written by Lance Martin, Brad Abrams, Isabella He, and Ben Lehrburger. Cache rules, anti-pattern counts, effort prices, and the hillclimb figures are theirs. The stills are from that article.