Lesson 3.2
Streaming and Real-time UX
How Claude streams responses, what the event stream contains, and how to design gateways and front ends that turn streaming into a responsive, resilient user experience.
Streaming is not an optimisation to add later. For any interactive Claude product it is the default transport, because perceived latency is governed by time to first token rather than total generation time, and because long outputs with thinking enabled can exceed ordinary HTTP timeouts if you wait for a complete response. The exam expects you to know the stream's structure and the design consequences of carrying it through your own infrastructure.
The event stream
With stream enabled, the API returns server-sent events in a fixed grammar:
| Event | Meaning | Typical handling |
|---|---|---|
message_start | Response begun; carries the message ID and initial usage | Create the response record |
content_block_start | A new block (text, tool_use, thinking) begins | Open a UI element or a buffer |
content_block_delta | Incremental content: text_delta, input_json_delta, thinking_delta | Append to the open block |
content_block_stop | Block complete | Finalise, parse accumulated JSON for tool input |
message_delta | Top-level changes such as stop_reason and output usage | Decide whether to continue a tool loop |
message_stop | Stream complete | Close the response |
ping | Keep-alive | Ignore |
error | Mid-stream failure such as overload | Surface and retry per policy |
Tool inputs arrive as partial JSON fragments across deltas; you accumulate them and parse at content_block_stop. The official SDKs provide helpers that assemble the final message for you while still exposing events, which is the recommended path; reimplementing the assembly is a common source of bugs.
Designing the gateway
Most enterprises do not let browsers call the model directly. The gateway receives the stream and re-emits it to the client, which raises design questions:
- Transport to the client. SSE is the natural fit and works through most proxies; WebSockets suit bidirectional chat with interruption. Confirm that load balancers and CDNs do not buffer responses, or the stream becomes a wait.
- Backpressure. A slow client must not stall the upstream read or hold a worker thread. Buffer bounded amounts and drop the connection if the client falls too far behind.
- Transformation. The gateway can filter events (hide thinking from end users, show it to internal users), inject status messages for tool calls, and redact content before it reaches the client.
- Logging. Record the assembled final message and usage, not every delta.
UX patterns
- Render text as it arrives; users tolerate long answers when they see progress.
- While the model is thinking, show an explicit "reasoning" state. With display of thinking omitted by default on current models, the stream may be silent for a while; a spinner that says what is happening prevents abandonment.
- Show tool calls as status lines ("Searching the knowledge base...") built from
content_block_startevents for tool_use blocks. Users understand waiting for a lookup. - Support interruption: let the user cancel, abort the upstream request, and record the partial response.
- For structured outputs streamed as JSON, render progressively only if the UI can handle partial objects; otherwise buffer and show a progress indicator.
Streaming tool inputs
Large tool inputs, such as a file the model is writing, can be streamed eagerly as they are generated rather than buffered server-side. When you enable this on a tool, the client owns validation: the accumulated input may be incomplete if max_tokens was reached, or invalid. Validate against the schema before executing, return a tool error if it fails, and check the stop reason before running any tool. Leave it off for non-streaming requests and for server tools.
Failure handling
A stream can fail after partial output: an overload error, a dropped connection, a client timeout. Streams are not resumable; the retry is a new request. Design the client to display partial output clearly as incomplete, retry idempotently where the action is read-only, and avoid retrying automatically where a tool call with side effects may already have executed. Set SDK and gateway timeouts so that the idle gap between events, not the total duration, is what triggers a timeout; a thinking phase can legitimately produce no visible deltas for a while.
Streaming in agentic systems
Agent runs are long, and streaming is how you keep humans informed. Stream status events per turn (which tool, which step, token usage so far), not raw transcripts. The Agent SDK and Managed Agents expose event streams for this purpose; your own loops should emit equivalent structured events. For batch or background jobs, streaming to a log or progress record replaces streaming to a user, but the practice is the same: never run a long task silently.
Testing streaming
Test with slow networks, dropped connections, very long outputs and tool-heavy turns. Assert that the assembled message equals what a non-streaming request would return, that usage is captured from message_delta, and that cancellation releases upstream resources. Load test the gateway for concurrent long-lived connections, which stress connection pools differently from short requests.
Exam focus
- Streaming lowers perceived latency via time to first token and is the practical default for interactive use and long outputs.
- Know the event sequence: message_start, content_block_start, content_block_delta, content_block_stop, message_delta, message_stop, with ping and error.
- Tool inputs stream as partial JSON; accumulate and parse at block stop, or let SDK helpers assemble the final message.
- Gateways must avoid buffering, handle backpressure, and can filter or transform events (for example hiding thinking from end users).
- Streams are not resumable; retries are new requests, so side-effecting tool calls need idempotency.
- Timeouts should be based on idle gaps between events, since thinking phases can be silent.
Check yourself
5 questions
1.A front-end team is implementing a streaming client that must render text as it arrives and also execute tool calls the model makes. Their first attempt tries to parse the tool input from each streamed event and fails because the JSON is incomplete. Which handling is CORRECT?
2.A streaming chat service logs that some conversations end with a half-written answer and no exception in the HTTP layer. Investigation shows the HTTP status was 200 and the connection closed normally. What is the MOST likely cause, and what should the client do?
3.A document-generation job requests very long outputs (tens of thousands of tokens) with a non-streaming call. Requests intermittently fail with client-side timeouts even though the API would eventually produce the response. Which change is MOST appropriate?
4.A coding assistant streams responses, and when the model writes a whole file through a tool the user sees nothing for many seconds and then the entire file at once. The team enables eager input streaming on that tool so file contents stream as they are generated. Which consequence MUST the team now handle?
5.A team enabled streaming in their backend, but users still see the whole answer appear at once after a delay. Network traces show the backend receives events incrementally from the API, yet the browser receives one large response at the end. What is the MOST likely cause?