Tech Analysis

Why LLM Prices Fell 80%: The Cache Is the Bill

October 2026

A price war with a surprising lever

Since June 2026, OpenAI and Anthropic have cut some of their API prices by as much as 80 percent, and a wave of cheaper open-weight models has pushed in the same direction. The headline cuts are easy to read. The more interesting part is where the savings actually come from, because for the fastest-growing kind of AI usage, coding and other agents, the number that matters is not the price of input or output. It is the price of reading the cache. This piece explains why, what is driving the cuts, and why cheaper tokens don’t always mean a cheaper bill.

What changed

OpenAI, 30 July. It cut GPT-5.6 Luna, its smallest model, by 80 percent to $0.20 per million input tokens and $1.20 per million output tokens, and the mid-range Terra by 20 percent. The flagship Sol didn’t change. OpenAI tied the cut to efficiency improvements in how it serves the models, and the move came only weeks after launch, which is earlier than price cuts usually arrive, according to Axios.
Anthropic, summer. It scrapped a planned increase and kept introductory pricing for Claude Sonnet 5. Later reports say its Opus 5.5 model costs substantially less than Opus 5, with Anthropic saying it needs less compute to serve, and that its Haiku 5.5 costs about a tenth of its predecessor for most requests.
Anthropic, 1 September. Per analyses of the cuts, it lowered the price of cache reads on its top model by 75 percent, from $1 to $0.25 per million tokens.
Open-weight rivals. Cheap open-weight models are hosted by many competing providers. One summary counts 29 for a single DeepSeek model on OpenRouter, which pushes prices towards the cost of serving.

Why the cache is the bill

To see why the cache price matters, look at how an agent works. A coding agent doesn’t send one question and get one answer. It runs in a loop: read a file, run a command, read the output, decide the next step. On every turn it sends the whole conversation so far back to the model, including all the earlier tool results. Only the newest slice is genuinely new text. Everything before it is a repeat.

Prompt caching exists for exactly this. When a model reads a prompt, it computes internal attention state for each token, often called the KV cache. If the start of a new request is identical to a recent one, the provider can reuse that stored state instead of recomputing it, and it bills those tokens at a much lower “cache read” rate. The effect is large, because in a long agent session nearly all the tokens are repeats.

Analyst Max Woolf, in an essay summarised across several write-ups, puts cache reads at 96 to 99 percent of token spend in agentic coding, with cache hit rates of around 90 to 100 percent. The arithmetic follows directly. If 97 percent of your bill is cache reads, cutting the cache read price by 75 percent cuts the total by about 73 percent, while changing input and output prices barely moves it. We couldn’t find the original essay, so the figures here rest on those summaries, and your own mix will differ.

What is pushing prices down

Cheaper serving. Both labs say they have made the models cheaper to run, and OpenAI explicitly credited serving efficiency. Gateway data cited in the analysis puts cost per token at about 43 percent of its April 2026 level.
Near-zero switching costs. Many applications talk to an OpenAI-compatible endpoint, so changing provider can be a configuration change. That limits any one lab’s pricing power.
Open-weight competition. Models such as DeepSeek and Kimi are good enough for many tasks at much lower prices, and routing tools make them easy to try.
Business pressure. Reporting says companies were pushing back on AI bills, with some adding budget caps or scaling back agent projects.

There is a financial backdrop too. Both companies are reported to be preparing public listings, and some reports put inference profit margins high even after the cuts. Those margin figures are reported estimates, not company disclosures, so we wouldn’t build conclusions on them.

Cheaper tokens, not always cheaper bills

A falling price per token and a falling bill are different things. Agents use far more tokens per task than a chat does: more turns, more context, more retries. One analysis of the July cuts noted that token prices have fallen steadily for a year while the cost of finishing a given task has gone up, so a headline price cut isn’t automatically a budget cut. That is the same point we made in our piece on why cloud AI bills are so hard to budget: the unit price is only one factor in the total.

What it means in practice

Measure your cache hit rate. If a large share of your spend is cache reads, that line item is the one to watch, and to compare between providers.
Route by difficulty. Send easy tasks to a small cheap model and keep the expensive one for work that needs it.
Keep your own evaluations. You need your own tests to know whether a cheaper model is good enough for your task.
Stay portable. Avoid building so tightly around one provider that moving is hard. Prices have changed several times in a few months.
Re-check assumptions regularly. A cost model from six months ago is probably wrong. Subscription quotas may be a more likely place to see value cut than token prices.

What we don’t know

Whether the cuts hold. Open-weight competition suggests pressure to stay low, but nobody can promise prices won’t rise.
What the real margins are. Reported figures come from unnamed sources and analysts.
How much of the saving reaches users who don’t use caching heavily.
The current price list. Providers change prices often, so check the official pricing pages before you budget.

Sources

For disclosure: we use AI tools, including Claude, to help research and draft our posts, and Anthropic, which makes Claude, is one of the companies discussed. We check the facts and sources ourselves, but many figures here come from secondary summaries and may have changed. We are not financial advisers, and this is general information, not advice for your budget.

More from Offgrid Studio

Why Cloud AI Bills Are So Hard to Budget
Running an AI Model Bigger Than Your RAM: What Works
NVIDIA OpenShell: A Sandbox for AI Agents, Explained

Back to the blog