Prompt caching: why a coffee break in your AI chat costs more than the question

What this is about

When you work with Claude Code, Codex or your own agent, every step sends the entire history back to the model: system instructions, tool descriptions, files read and all previous messages. In a longer session, that quickly adds up to 50,000 to several hundred thousand tokens per request, even though only a few lines are new. Whether you pay full price for that or almost nothing comes down to the prompt cache.

This leads to a seemingly paradoxical result: a short question after an hour's break can cost many times more than the same question asked two minutes earlier. One Reddit post mentions a factor of 20. We couldn't view the original thread, but the number follows directly from Anthropic's price list, as we calculate below. At the same time, the major providers have tweaked their cache pricing in recent weeks, so some overviews online are out of date.

What's actually in the cache

When a language model reads text, it computes intermediate results for every token in every layer, called keys and values. That's how the model "remembers" what came earlier in the text. The provider can store this KV cache. If the next request starts with exactly the same text, the prefix, the server skips the computation and loads the stored values instead. That's faster and cheaper.

The catch: each token's values depend on all the tokens before it. Change a single character at position 3,000, and everything after it is invalid. It's like a recipe where you swap an ingredient in step 2: you have to redo every step that follows. Microsoft puts it bluntly:

A single character difference in the first 1,024 tokens will result in a cache miss.

Three terms determine the bill: - TTL (time to live): how long the cache survives without being used. Each hit usually resets the clock. - Write premium: creating a cache often costs more than the normal input price. - Read discount: hits cost only a fraction of the input price.

What breaks the cache, even without a break

Expiry is only one cause; the other is invalidation. According to Anthropic, the order is tools → system prompt → messages. If the tool definitions change, everything is gone. Turning web search on or off wipes the system prompt and message caches. OpenAI's docs also list model, tools, text.format (the response schema), reasoning.effort, text.verbosity and compaction.

For Claude Code, the official docs list: switching models (each model has its own cache), changing the effort level on most models, enabling fast mode, connecting an MCP server whose tools are preloaded, automatic pruning of old images, and /compact. Compaction replaces the history with a summary and inevitably builds a new prefix. Editing CLAUDE.md mid-session, on the other hand, keeps the cache intact, but the change only takes effect after /clear, /compact or a restart. /rewind jumps back to an already cached state, so it's cheap.

Common pitfalls in practice: - Timestamps in the prompt: An author on Dev Genius reports that a single line with a timestamp pushed his agent's cost per call from $0.0328 to $0.0815. This hasn't been independently confirmed. - Strict JSON schema: In July, a developer on the OpenAI forum reported that GPT-5.6 Terra with response_format: json_schema produced no cache writes at all. According to OpenAI support, it worked in a retest in August. One user confirmed this with explicitly set breakpoints but saw a delay before the cache became readable. - Small prompts: In a test on dev.to, a prompt of about 1,300 tokens was never cached by GPT-5.5 across 15 attempts, even though it exceeded the minimum length. And even with a warm cache, roughly one in four calls paid full price. All providers describe caching as "best effort," meaning there's no guarantee.

Where the factor of 20 comes from

On a Claude subscription (Pro, Max, Team, Enterprise), Claude Code uses a 1-hour cache for the main conversation. With an API key, through cloud providers, or on usage credits after hitting your limit, it's only 5 minutes. Subagents always get 5 minutes unless you configure otherwise. A 1-hour write costs 2 times the input price, a hit 0.1 times. So if you come back after more than an hour, your next question costs 2 instead of 0.1, exactly 20 times as much. With Opus 5.5 and Sonnet 5.5 (reads: 0.05x), it's 40; with Fable and Mythos 5.1/5.2 (0.025x), it's as much as 80.

Blogger J. D. Hodges ran the numbers on a 586,000-token session: at Fable 5.1 list prices, it costs about $0.15 from a warm cache and $11.72 to rewrite. On a subscription, that doesn't show up as a bill, but it eats into your usage limit. Claude Code explicitly warns about this for sessions over 100,000 tokens after about an hour of inactivity. Since version 2.1.299 (October 14), Claude Code summarizes conversations over 200,000 tokens after roughly 54 minutes of inactivity, before the cache expires ("idle compaction"). You can turn this off with "idleCompaction": false.

An author on wmedia.es analyzed three of his own sessions: rereading the history accounted for 65, 39 and 54 percent of costs, even though it's billed at the discounted rate.

What's changing right now

With GPT-5.6 (July 2026), OpenAI ended the "automatic and free" model. It's still automatic, since OpenAI sets an implicit breakpoint at the end of the last message by default. What's new are explicit breakpoints (up to four writes per request, i.e. markers for how far to cache) and a write fee. A write with a minimum lifetime of 30 minutes costs 1.25 times the input price. Since October 20, the Responses API has also offered the 12-hour cache at 2.5 times, not 1.25 times as often reported. According to OpenAI, that's the total price, not an additional surcharge. The November 1 date comes from Microsoft's Azure docs: 12-hour writes are billed there from that day. Just how confusing this is shows in the widely used billing library LiteLLM. It initially booked 12-hour writes at the 30-minute rate, displaying only about half the real cost. A fix in the main branch now bills them correctly.

Google is doubling the prices of Gemini 4 Argon and Carbon on December 1: input goes from $2 to $4, cache reads from $0.10 to $0.20 per million tokens. That's in the official price list. So the discount stays at 95 percent, but it still gets more expensive. According to Licenseware, an introductory discount on Vertex AI is expiring. Gemini 4 only supports implicit caching from 4,096 tokens. There's also a quirk: if a new request adds fewer than 2,050 tokens, the last 2,050 tokens of the previous request are billed at full price.

On November 13, DeepSeek brought features from its own app into the API. The new line items cover tools, not the cache: web search and web fetch cost $0.75 per 1,000 calls each, file parsing and OCR $0.225 per 1,000 pages. On top of that come text-to-speech (TTS) and speech recognition (ASR), both currently at half price. The disk-based cache remains very cheap and, according to the docs, is usually available for "a few hours to days."

Fact-checked: cache rules at a glance

As of November 15, 2026, based on official docs and pricing pages. Multipliers refer to the normal input price.

Worked example: when the long-lived cache pays off

We ran the numbers ourselves at list prices: an agent has a fixed 50,000-token prefix and makes 100 runs at evenly spaced intervals. We only count the cost of the prefix and assume every possible hit succeeds. For OpenAI's 12-hour cache, we conservatively assume a new write every 12 hours.

Three takeaways: 1. The long cache only pays off if your breaks are longer than the short TTL but shorter than the long one. With a 20-minute break, Anthropic's 1-hour cache saves about 94 percent compared with the 5-minute cache. 2. If the cache keeps expiring, caching costs more than no caching at all. You keep paying the write premium but never get the read discount. That matches the Dev Genius observation that an expired 30-minute cache was "worse than no caching at all." 3. OpenAI's break-even point is the third request, according to Cellcog's calculation: one write plus two hits cost 2.7 times the input price, three requests without caching 3 times.

For Gemini 4 Argon, 100 hits currently cost about $0.60, and about $1.19 from December. But Google doesn't document how long the implicit cache survives breaks.

What you can do

• In Claude Code, run /usage and check the "Prompt cache (main)" line. It shows your hit rate, misses and often their likely cause.
• Set your model and effort level at the start, and don't switch mid-task.
• Run /compact before a longer break, while the cache is still warm, not after. Use /clear for a new topic.
• If you work with an API key and often take 5- to 60-minute breaks, you can turn on the long cache with promptCacheTtl: "1h" or CLAUDE_CODE_PROMPT_CACHE_TTL=1h.
• In your own agents, put fixed content at the start and changing content like the date or username at the end, and only append to the history, never rewrite it.

Our verdict

Our take: providers are clearly converging on Anthropic's model: a short TTL by default, a premium for long-lived caching, a write fee, and in return very steep read discounts of 90 to 97.5 percent. For you, that means the list price per million tokens says less and less about what agents actually cost. What matters is your hit rate and the rhythm of your breaks. DeepSeek and xAI are the low-cost outliers with no write fee, but neither formally documents how long its cache lasts. Google doesn't document the TTL for Gemini 4 at all. The lack of transparency remains a problem: invalidation rules now fill entire documentation pages, billing tools have already gotten it wrong, and on subscriptions you only see cache costs indirectly, as a shrinking usage limit.