Data, Memory and State Design

The Messages API is stateless: every request carries everything the model will know. That one fact shapes all state design. You, not the platform, decide what is remembered, where it is stored, how it is summarised and when it is deleted. Agent runtimes such as the Agent SDK and Managed Agents add session persistence on top, but the design responsibility stays with the architect.

Four kinds of state

State	What it holds	Where it typically lives	Lifecycle
Conversation state	The messages array for the current interaction	Your application store; a session in the Agent SDK or Managed Agents	Lives for the session; compacted or truncated as it grows
Working context	Instructions, retrieved passages, tool results currently in the window	The request itself	Rebuilt every call; shaped by caching and context editing
Long-term memory	Facts, preferences, learned procedures that outlive a session	Memory tool files, a database, a memory store	Curated; needs write policy and expiry
Task state	Plans, checkpoints, partial results of a long job	Filesystem, database, task board	Lives for the task; enables resume after failure

Keeping these separate avoids the classic mistake of treating the conversation transcript as the system of record for everything.

Managing the context window

Three mechanisms exist, and they are not interchangeable:

• Compaction summarises earlier conversation when the context approaches a threshold. Where it is offered server-side, the API returns a compaction block that you must pass back unchanged on the next request; appending only the text of a response silently discards the compaction state. Compaction preserves meaning while losing detail.
• Context editing removes content rather than summarising it, for example clearing old tool results or thinking blocks. It is the right tool when tool outputs are bulky and transient, such as search results that have already been digested.
• Client-side summarisation is your own code summarising and rewriting history. It gives you full control over what is kept, at the cost of owning the quality of the summary.

Choose compaction for long dialogues where continuity matters, context editing for tool-heavy loops, and client-side summarisation when regulatory or product requirements dictate exactly what persists. Current server-side options and their beta status are listed in the Anthropic docs.

Long-term memory

The memory tool pattern gives the model a file-like store it can read and write across sessions. It is powerful and dangerous in equal measure: an agent that can write its own memory can also write itself a bad habit, or be induced to by an injected instruction. Design rules:

• Scope memory per user or per tenant; never share memory across trust boundaries.
• Separate facts (checked, stable) from notes (the agent's working observations) so that a reviewer can prune the latter.
• Define expiry and review. Memory that is never pruned becomes stale context that costs tokens every call.
• Treat memory contents as untrusted input when they are read back, exactly like a tool result.

For enterprise retrieval, long-term knowledge usually belongs in a governed store with access control, with the model retrieving per request, rather than in agent memory.

Resumable tasks

Long jobs fail: rate limits, timeouts, a tool outage. Design for resume rather than restart. Persist the plan and completed steps outside the model's context; on failure, reload them and continue. In Agent SDK designs the session can be resumed by ID; in your own loops, write a checkpoint after every side-effecting step. Side effects themselves should be idempotent, so that a replayed step does not double-create a record.

Data classification and minimisation

Every piece of data that enters a prompt is data you have transmitted to the model provider under the terms of the platform you chose. Apply minimisation:

• Send the fields the task needs, not whole records. A refund decision needs the order and the policy, not the customer's full profile.
• Pseudonymise identifiers where the model does not need the real value, and re-identify in code afterwards.
• Decide what you log. Request and response logs are often the largest and least governed store of personal data in the system.
• Know the retention regime of your platform and whether zero-data-retention applies to your organisation, and note that some features and the newest model tiers may not be available under it.

Residency requirements shape the platform decision and, where offered, inference location controls. Confirm current options in the docs rather than assuming.

A reference design

request -> assemble context:
             [cached system prompt + tool defs]
             [retrieved passages, access-filtered]
             [compacted history]
             [current user turn]
        -> model
        -> tool calls -> results (logged, size-capped, cleared when stale)
        -> response
        -> persist: history delta, checkpoint, memory writes (reviewed)

Each arrow is a place to apply a policy: size caps on tool results, access filtering on retrieval, review on memory writes, retention on logs.

Exam focus

• The API is stateless; conversation, working, long-term and task state are distinct and should be stored differently.
• Compaction summarises and must have its compaction block passed back; context editing clears content; client-side summarisation gives full control. Know which fits which scenario.
• Memory must be scoped per user or tenant, pruned, and treated as untrusted when read back.
• Long tasks should checkpoint outside the context and use idempotent side effects so they can resume rather than restart.
• Data minimisation, pseudonymisation and log governance are architecture decisions, not afterthoughts.
• Retention and residency requirements influence platform choice; verify current zero-data-retention and inference location options in the docs.