Lesson 2.2
Context Engineering Principles
How to treat the context window as a scarce resource, decide what enters it and when, and structure it so that attention, caching and cost all work in your favour.
Prompt engineering asks how to phrase an instruction. Context engineering asks a bigger question: of everything the system could put in front of the model on this turn, what is the smallest set of high-signal tokens that makes the right outcome likely? For agents that run for many turns and read many tool results, this question dominates quality, cost and reliability. The professional exam treats it as an architectural discipline.
Context is a budget, not a bucket
Large context windows invite the belief that more is better. In practice, attention is finite: as the window fills, the model's ability to recall and weigh any individual fact degrades, a phenomenon often described as context rot. Every token you add competes with every other. The architect's job is to curate.
A useful inventory of what competes for the budget:
| Component | Tends to grow because | Control |
|---|---|---|
| System prompt | Rules accumulate after every incident | Periodic pruning; test deletions against evals |
| Tool definitions | Every integration adds tools | Minimal, non-overlapping tools; deferred loading |
| Examples | "Add one more case" | Few, diverse, canonical examples |
| Retrieved content | Top-k creep | Tight k, relevance thresholds, summarised passages |
| Conversation history | Time | Compaction, context editing, summarisation |
| Tool results | Verbose APIs | Size caps, field selection, clearing when digested |
Right-altitude instructions
System prompts fail at two extremes. Too specific, and you get brittle if-else logic that breaks on the first unanticipated input. Too vague, and the model lacks the signal to act consistently. Aim for the altitude of a good brief to a capable new colleague: state goals, constraints, the shape of a good result and the heuristics that distinguish good from bad, then trust the model to apply them. Organise with clear sections (background, instructions, tool guidance, output format) so the model and your reviewers can find things.
Newer models are more capable of judgement, and prompts written for older models are often too prescriptive. Audit prompts when migrating; over-direction now costs quality.
Tools as context
Tool definitions are tokens too, and poorly designed tool sets are a leading cause of agent failure. Each tool should have a clear, non-overlapping purpose, a description that explains when to use it and when not to, and a schema with unambiguous parameter names. If a human engineer could not tell which of two tools to call, neither can the model.
When the catalogue is large, do not load it all. Tool search with deferred loading lets the model discover tools on demand from a short description, keeping the always-present set small. Agent Skills work similarly for procedures: a skill's metadata is always visible, its full instructions load only when relevant.
Just-in-time versus pre-loaded
Two retrieval strategies exist. Pre-loading embeds relevant data in the prompt before the model runs, which is predictable and cache-friendly for stable corpora. Just-in-time retrieval gives the model tools (search, file read, database query) and lets it pull what it needs, which scales to large or changing data and keeps the context lean, at the cost of extra turns.
Production systems usually blend them: pre-load what is small and stable (policies, schema descriptions), retrieve what is large or dynamic, and let the model follow references rather than dumping everything up front.
Long-horizon techniques
Agents that run for hours cannot keep everything in the window. Three techniques, often combined:
- Compaction condenses history into a summary that preserves decisions, open questions and key facts. Server-side compaction is available on supported models; otherwise implement it in your loop.
- Structured note-taking has the agent maintain an external file (a plan, a progress log, a decisions record) that it re-reads. The notes survive compaction and give a human a place to intervene.
- Subagent isolation delegates reading-heavy subtasks to workers that return compact findings, so the orchestrator's context holds conclusions rather than raw material.
Ordering for attention and caching
Order the context deliberately. Stable content first (system prompt, tool definitions, reference documents), then the evolving conversation, then the current query last. This serves two masters at once: models tend to attend well to instructions placed after long documents, and prompt caching is a prefix match that rewards a stable front and a volatile tail. Never interpolate timestamps, request IDs or user-specific data into the stable prefix.
Anti-patterns
- The kitchen-sink system prompt that documents every past incident.
- Twenty tools loaded on every request when three are used.
- Tool results left in history long after they were consumed.
- History passed between agents as raw transcript.
- Reaching for a bigger context window instead of curating.
Exam focus
- Context engineering means choosing the smallest high-signal set of tokens for each turn; more context is not automatically better.
- Instructions should sit at the right altitude: goals, constraints and heuristics, not exhaustive branching rules.
- Tool definitions are context; keep them minimal and non-overlapping, and use deferred loading or skills for large catalogues.
- Blend pre-loading (small, stable data) with just-in-time retrieval (large, dynamic data).
- Long-horizon agents rely on compaction, external structured notes and subagent isolation.
- Order context stable-first and volatile-last to serve both attention and prompt caching.
Check yourself
5 questions
1.A field-service assistant preloads 150,000 tokens of equipment manuals into every request so that any question can be answered. Latency is high, costs are rising, and answer quality on specific questions has not improved compared with a smaller context. Which context engineering change is MOST appropriate?
2.An enterprise agent integrates with twelve internal systems and now exposes more than 80 tools in every request. Tool definitions alone consume tens of thousands of tokens, the model sometimes picks a wrong but similarly named tool, and each new integration makes the problem worse. Which approach BEST addresses the problem without removing capabilities?
3.A data team's system prompt has grown to include detailed procedures for twenty different report types, although any given request needs only one. The procedures are expertly written and the team does not want to lose them. Which design keeps the detail available while keeping the per-request context lean?
4.An operator needs to change an assistant's behaviour mid-conversation (for example switching to a terse mode or injecting updated account state) without losing the cached conversation prefix and without placing operator instructions where a user could forge them. The model in use supports mid-conversation system messages. Which approach is MOST appropriate?
5.An analytics agent's query tool returns the full result set as JSON, often 40,000 tokens, of which the model typically uses a handful of aggregate values. After a few queries the context is dominated by raw rows, the agent slows down and starts making arithmetic mistakes. Which combination of changes BEST fixes this?