Prompt Caching for Developers

Prompt caching lets the API reuse the processed form of a prompt prefix across requests instead of re-reading it every time. For agents, chat apps and document Q&A it is the biggest cost and latency saving available, and it is also the feature most often configured in a way that silently does nothing. The exam tests the mechanics precisely.

The one rule

Caching is a prefix match. The request is rendered in a fixed order, tools then system then messages, and a cache_control marker says "cache everything up to and including this block". On the next request, the API looks for an identical byte-for-byte prefix. Any change before a breakpoint, even one character, invalidates that breakpoint and every breakpoint after it. Design the prompt so stable content comes first and volatile content comes last.

Marking a breakpoint

Add cache_control: {"type": "ephemeral"} to a content block. The most common placement is the last system block:

resp = client.messages.create(
    model=MODEL, max_tokens=1024,
    system=[{"type": "text", "text": LONG_INSTRUCTIONS_AND_DOCS,
             "cache_control": {"type": "ephemeral"}}],
    messages=[{"role": "user", "content": question}],
)
print(resp.usage.cache_creation_input_tokens, resp.usage.cache_read_input_tokens)

const resp = await client.messages.create({
  model: MODEL, max_tokens: 1024,
  system: [{ type: "text", text: LONG_INSTRUCTIONS_AND_DOCS,
             cache_control: { type: "ephemeral" } }],
  messages: [{ role: "user", content: question }],
});

Because tools render before system, a breakpoint on the last system block caches the tool definitions too. Markers can go on system text blocks, tool definitions, and message content blocks including text, image, document, tool_use and tool_result. You may set up to four breakpoints per request, which lets you cache sections that change at different rates: tools (never), a document (daily), the conversation so far (per turn).

Automatic caching

Instead of placing markers yourself, you can pass cache_control as a top-level request field. The API places a breakpoint on the last cacheable block and moves it forward as the conversation grows. This is the simplest option for multi-turn chat. Use explicit markers when the prompt ends with unique per-request content, since an automatic breakpoint after that content writes a cache entry nobody will ever read.

Multi-turn conversations

Put a breakpoint on the last block of the newest turn each time. Earlier breakpoints remain valid read points, so each request reads the whole prior conversation from cache and writes only the new turn. The healthy signature in usage is a large, growing cache_read_input_tokens and a small cache_creation_input_tokens.

Lifetime and pricing

Setting	Lifetime	Write cost	Read cost
{"type": "ephemeral"}	5 minutes, refreshed on every read	Premium over base input (about 1.25x)	Small fraction of base input (about 0.1x, less on some models)
{"type": "ephemeral", "ttl": "1h"}	1 hour, refreshed on every read	Higher premium (about 2x)	Same as above

The timer restarts whenever the entry is read, so continuous traffic keeps a 5-minute cache alive indefinitely. Choose the 1-hour TTL only when gaps between requests sharing the prefix are routinely longer than 5 minutes. Exact multipliers are on the pricing page; the shape of the trade-off is what matters: two requests inside the window already pay back the write premium.

Minimum length

Prefixes shorter than a model-specific minimum (on the order of a thousand tokens, varying by model) are not cached even when marked. There is no error; cache_creation_input_tokens is simply zero. Check the model's minimum before concluding caching is broken.

Verifying it works

The usage object is the only ground truth:

• First request: cache_creation_input_tokens > 0, cache_read_input_tokens = 0.
• Second identical-prefix request: cache_read_input_tokens > 0.

If reads stay at zero, something in the prefix is changing. The usual suspects: a timestamp or user name interpolated into the system prompt, a request ID early in the content, tool definitions serialised in non-deterministic order or varying per user, conditional system sections, or switching model (caches are per model and per workspace). Fix by moving the dynamic piece after the last breakpoint or making it deterministic. Add an integration test asserting that a second request reads from cache, because caching regressions are silent and only show up on the invoice.

Where caching fits

• Agents: cache the system prompt and tools once, then the growing history each turn.
• Document Q&A: cache the document block; vary only the question.
• Batches: a shared system block with a marker is read from cache across the batch.
• Few-shot prompts: cache the examples; they rarely change.

Cache reads on most models also do not count against input token rate limits, so caching raises effective throughput as well as cutting cost.

Exam focus

• Caching is a prefix match over tools, then system, then messages; any change before a breakpoint invalidates it and everything after it.
• cache_control: {"type": "ephemeral"} marks a breakpoint; up to four per request; a top-level cache_control enables automatic placement.
• Default TTL is 5 minutes, refreshed on each read; ttl: "1h" costs more to write and suits gaps longer than 5 minutes.
• Cache writes cost more than base input, reads cost far less; prefixes below the model's minimum token length are silently not cached.
• cache_read_input_tokens greater than zero on repeat requests is the proof caching works; zero means a volatile prefix.
• Caches are scoped per model and per workspace and never shared across organisations.