The Context Window and Token Budgeting

The context window is the total number of tokens a single request can hold: everything you send plus everything the model generates in response. It is the hardest constraint in any Claude architecture, because exceeding it is an error and approaching it degrades quality and raises cost. The exam tests what consumes the window, how to measure consumption, and how to budget it deliberately.

What a token is

Tokens are the units the model reads and writes. In English prose a token averages a little under four characters, so a thousand tokens is roughly 750 words; code, non-Latin scripts and numbers tokenise less efficiently. Tokenisation differs between model generations, so counts are not portable across models. Never estimate tokens with a third-party tokeniser; use the API's own counter.

What consumes the window

Every request sends the full picture, and all of it counts:

Component	Notes
System prompt	Sent every request; usually stable
Tool definitions	Sent every request; dozens of tools can cost thousands of tokens
Conversation history	Grows every turn; tool results are often the largest part
Documents and images	Images cost roughly their pixel area divided by 750; PDFs cost text plus a per-page image
Thinking	Counts towards max_tokens and the window for the current request
Output	Bounded by max_tokens

The window must accommodate input plus max_tokens: a request whose input is near the limit with a large max_tokens is rejected before generation starts. Current models offer windows of several hundred thousand tokens, with larger windows available on some; the exact figures are on the models page of the documentation and should not be memorised as fixed.

Measuring

Two tools give exact numbers. The token counting endpoint (client.messages.count_tokens) accepts the same system, messages and tools you would send and returns the input token count without running the model; it is free apart from rate limits and is the right way to check a request before sending it, to size chunks, or to decide whether to compact. The usage object on every response reports input_tokens, output_tokens and the cache fields cache_creation_input_tokens and cache_read_input_tokens; log it on every call.

count = client.messages.count_tokens(model=MODEL, system=SYSTEM, tools=TOOLS, messages=messages)
if count.input_tokens + MAX_OUTPUT > WINDOW * 0.8:
    messages = compact(messages)

In Claude Code, /context shows how the current window is divided between system prompt, tools, memory files, messages and free space, and /cost shows spend for the session.

Budgeting

Think of the window as four regions and allocate each a ceiling:

1. Fixed prefix: system prompt and tools. Keep it stable (for caching) and lean (every token here is paid on every request). Trim tool descriptions to what the model needs; move rarely used tools behind tool search or into subagents.
2. Working history: recent turns and tool results. Decide the policy for shrinking it before it is needed: summarise, clear old tool results, or hand off to a fresh context.
3. Retrieved or attached content: documents, search results. Retrieve less, more precisely; chunk long documents and send the relevant chunks, not the whole file.
4. Output reserve: max_tokens plus thinking. Reserve it up front so that the input policy never squeezes it out.

A practical rule is to trigger compaction or clearing at around 70 to 80 percent of the window rather than at the limit, because quality on long-context tasks declines before the hard error arrives.

Long-context technique

When a request legitimately needs a lot of material, placement matters. Put long documents at the top of the prompt and the instructions and question at the end; the documented improvement in response quality is substantial. Wrap documents in tags with metadata, and ask the model to extract relevant quotes before answering. For multi-document tasks, index the documents so the model can refer to them unambiguously.

Cost follows the window

Input tokens are billed per request, so a conversation that resends 150,000 tokens of history every turn costs that much every turn. Prompt caching reduces the price of a stable prefix dramatically but does not reduce the count, and nothing it does helps if the prefix changes. Context discipline is therefore the primary cost control, ahead of model choice.

Exam focus

• The context window holds input plus output for one request; max_tokens must fit inside it alongside the input.
• System prompt, tool definitions, history, documents, images and thinking all count; tool results are typically the fastest-growing component.
• count_tokens measures a request before sending; usage reports what each response consumed; neither should be replaced by a third-party tokeniser.
• Budget the window as fixed prefix, working history, retrieved content and output reserve, and compact well before the hard limit.
• Long documents go at the top, the question at the end, with quote extraction before the answer.
• Caching lowers the price of a stable prefix but not its size; shrinking context is the first cost lever.