Prompt Caching

Prompt caching lets the API reuse the processed form of a prompt prefix across requests instead of recomputing it. For any application that sends the same system prompt, tool definitions, examples or documents repeatedly, it is the single largest cost and latency lever available. Because it is a prefix match on exact bytes, it is also easy to defeat by accident. The exam tests the mechanism, the economics and the invalidation rules.

The mechanism

You mark a point in the request with cache_control: {"type": "ephemeral"}. Everything from the start of the request up to that block becomes a cacheable prefix. The request is rendered in a fixed order, tools, then system, then messages, so a breakpoint on the last system block caches the tools and the system prompt together.

response = client.messages.create(
    model=MODEL, max_tokens=1024,
    tools=TOOLS,
    system=[
        {"type": "text", "text": POLICY_AND_EXAMPLES,
         "cache_control": {"type": "ephemeral"}},       # breakpoint
    ],
    messages=[{"role": "user", "content": question}],   # varies per request
)

On the first request the prefix is written to the cache; on later requests whose prefix is byte-identical it is read. Up to four breakpoints may be set per request, which allows separate boundaries for tools, system, long documents and conversation history. A top-level cache_control on the request enables automatic caching, which places a moving breakpoint at the end of the latest cacheable block; it suits multi-turn conversations, while explicit markers suit prompts with a shared prefix and a unique tail.

A prefix shorter than the model's minimum cacheable length is silently not cached. The minimum varies by model (from a few hundred to a few thousand tokens), so a short system prompt may show no cache activity for that reason alone.

Economics

Cache writes cost more than normal input; cache reads cost much less. With the default five-minute lifetime a write costs about 1.25 times the base input price and a read about one tenth, so the second request already pays for the first. A one-hour lifetime is available ("ttl": "1h"), at roughly double the write cost, for traffic with gaps longer than five minutes between requests. Every read refreshes the lifetime, so steady traffic keeps a five-minute cache warm indefinitely. Consult the pricing page for exact multipliers per model.

The usage object reports what happened: cache_creation_input_tokens (written this request), cache_read_input_tokens (served from cache) and input_tokens (the uncached tail). The success metric is reads. If reads stay at zero across requests that should share a prefix, something is invalidating it.

Placement

Pattern	Where the breakpoint goes
Stable system prompt and tools, varying questions	Last system block
Shared long document, many questions about it	End of the document block, before the question
Multi-turn conversation	Last block of the most recent turn (or automatic caching), so each turn reuses the whole previous prefix
Agent loop	One explicit marker after the static prefix plus automatic caching for the growing history

The rule in every pattern is the same: stable content first, breakpoint at the boundary, volatile content after it.

Invalidation

Any byte change before a breakpoint invalidates that breakpoint and everything after it. Changes are tiered: a change to the tool definitions invalidates tools, system and messages caches; a change to the system prompt invalidates system and messages; a change in messages invalidates only the messages cache from that point. Switching models invalidates everything, because caches are model-scoped. Changing thinking or effort settings invalidates the messages cache. Changing tool_choice does not disturb the tools and system caches.

Silent invalidators that defeat caching without any visible error:

• a timestamp, request ID or user name interpolated into the system prompt;
• non-deterministic JSON serialisation (unsorted keys) of tools or examples;
• a tool list that varies per request or is reordered;
• toggling thinking or effort between requests;
• editing earlier turns of the conversation instead of appending.

Diagnosis is mechanical: log two consecutive request bodies, strip the cache_control markers, and diff; the first divergence inside the shared region is the culprit.

Caching and the rest of the stack

Claude Code and the Agent SDK cache automatically, which is why a stable CLAUDE.md and tool set matter there too. Server-side tools such as web search insert their own cache writes after results. Message Batches can use caching but offer no timing guarantees, so hit rates are lower. And caching reduces price, not context: a cached 100,000-token prefix still occupies 100,000 tokens of the window.

Exam focus

• Caching is an exact-bytes prefix match; render order is tools, system, messages, and a breakpoint on the last system block covers tools too.
• Up to four cache_control breakpoints; prefixes below the model's minimum length are silently not cached.
• Writes cost roughly 1.25x base input (about 2x for the one-hour TTL); reads cost roughly 0.1x; reads refresh the lifetime.
• cache_read_input_tokens is the success metric; zero reads across identical-prefix requests means a silent invalidator.
• Tool changes invalidate everything, system changes invalidate system and messages, message edits invalidate from the edit onward; model switches invalidate all.
• Keep volatile content (timestamps, IDs, the user's question) after the last breakpoint.