Managing Context Across Turns

The context window is the total number of tokens the model can see in one request: system prompt, tools, every message, and the output it is about to write. Multi-turn apps and agents fill it relentlessly, and when it overflows the request fails. Managing it well is the difference between an agent that runs for an hour and one that dies after ten tool calls. The exam tests the strategies and their trade-offs.

What fills the window

Source	Growth pattern
System prompt and tool definitions	Fixed per request
User and assistant turns	Linear with conversation length
Tool results	Often the largest; a single file read or search can be thousands of tokens
Extended thinking blocks	Returned in content and resent if you echo them
Images and PDFs	Large fixed cost per attachment

Because the API is stateless, all of this is resent on every call, so cost grows with the square of the turn count unless you cache or trim.

Measure before you manage

Use count_tokens on the exact payload you are about to send, and compare with the model's context window from the models page. Leave headroom for max_tokens; input plus output must fit. In an agent loop, check after every tool result, not just at the start.

def within_budget(messages, limit):
    n = client.messages.count_tokens(model=MODEL, system=SYSTEM, tools=TOOLS,
                                     messages=messages).input_tokens
    return n + MAX_OUTPUT < limit

Client-side strategies

Sliding window. Keep the system prompt plus the most recent N turns. Simple and predictable, but forgets early decisions.

Token-budgeted truncation. Drop the oldest user/assistant pairs until count_tokens is under budget. Always remove whole pairs so roles still alternate and no tool_result is left without its tool_use.

Summarisation. When the window nears a threshold, ask Claude to summarise the oldest portion into a compact note that captures names, decisions, constraints and open items, then replace those turns with a single user message containing the summary. This preserves meaning at a fraction of the tokens.

summary = ask(f"Summarise this conversation so far for your own future reference, "
              f"keeping all facts, decisions and user preferences:\n{render(old_turns)}")
messages = [{"role": "user", "content": f"<summary>{summary}</summary>"},
            {"role": "assistant", "content": "Understood, I have the context."}] + recent_turns

Trim tool results at the source. Return only what the model needs: the first N lines of a file, the top K search hits, a status code instead of a full HTML page. Put size limits in the tool implementation. This is usually the biggest single win in agents.

Externalise memory. Store facts in a database or a file the agent can read on demand, instead of keeping them in the transcript. The memory tool and retrieval patterns follow this idea.

Server-side help

On supported models the API can do some of this for you:

• Context editing clears old tool results (and optionally tool inputs or thinking blocks) from the conversation before the model sees it, based on a strategy you pass in context_management.edits. It removes rather than summarises, so use it for bulky, stale tool output.
• Compaction summarises earlier context automatically when it approaches a threshold and returns a compaction block that you must echo back on subsequent requests, so always append the full response.content to history.

Both are beta features with their own headers; read the context editing and compaction pages on docs.anthropic.com for the current parameters and model support.

Interaction with caching

Every trimming strategy edits the prefix, which invalidates cached turns from the edit point onward. Trim in large, infrequent steps rather than a little every turn, keep the system prompt and tools untouched, and place a cache breakpoint on the latest turn so the untouched portion is still read from cache. Append-only histories cache best; mid-conversation operator instructions can be added as additional messages rather than by editing the system prompt.

Signs you need to act

• count_tokens approaching the window, or a 400 saying the prompt is too long.
• input_tokens on each call climbing faster than the conversation visibly grows (large tool results).
• The model forgetting instructions that were given early (dilution rather than overflow); move those into system.
• Rising cost per turn with no corresponding increase in value.

Exam focus

• The context window must hold system, tools, all messages, tool results and the output; the API is stateless so all of it is resent each turn.
• Measure with count_tokens on the real payload and leave room for max_tokens.
• Truncate in whole user/assistant pairs; summarise old turns into a compact note when meaning must be preserved.
• Limiting tool result size inside the tool implementation is the most effective control in agents.
• Context editing clears stale tool results server-side; compaction summarises and returns a block that must be echoed back.
• Editing the prefix invalidates the prompt cache from that point, so trim rarely and keep system and tools stable.