Prefilling, Stop Sequences and Output Control

Beyond the prompt itself, the Messages API offers a handful of parameters that control how generation proceeds and when it ends. Some are universal (max_tokens, stop_sequences), some are model-dependent (temperature, top_p, top_k) and one, assistant prefill, has moved from recommended technique to rejected request within a few model generations. The exam tests what each does, how the response reports which control fired, and which controls still apply to current models.

max_tokens

max_tokens is the hard ceiling on output tokens for one response, including any extended thinking. It is required on every request. When generation hits it the response has stop_reason: "max_tokens" and the output is truncated mid-sentence or mid-JSON. Set it from the task: a classification needs a few dozen tokens, a code refactor may need tens of thousands. Setting it too low is a common cause of broken structured output; setting it very high is harmless to cost (you pay for tokens generated, not permitted) but the SDKs require streaming for very long non-streaming requests to avoid HTTP timeouts.

Stop sequences

stop_sequences is a list of strings; when the model produces one, generation stops immediately. The response has stop_reason: "stop_sequence" and a stop_sequence field naming which one fired. The sequence itself is not included in the output.

response = client.messages.create(
    model=MODEL, max_tokens=2048,
    system="Reason inside <thinking> tags, then answer inside <answer> tags.",
    messages=[{"role": "user", "content": question}],
    stop_sequences=["</answer>"],
)
answer = response.content[0].text.split("<answer>")[-1].strip()

Typical uses: cut generation at the end of a structured section, prevent the model from continuing into a second example in a few-shot layout, or stop a list after a known terminator. Stop sequences are matched on the raw output, so choose strings that cannot appear legitimately in the content.

Sampling parameters

temperature (0 to 1, default 1) controls randomness: lower values make output more deterministic, which suits classification and extraction; the default suits creative work. top_p (nucleus sampling) and top_k offer alternative controls; the long-standing guidance is to alter temperature or top_p, not both. Two caveats for current models: extended thinking requires default sampling settings, and the newest model generations reject non-default values outright. Temperature 0 reduces variance but does not make output fully deterministic.

Prefill

Prefilling means ending the messages array with a partial assistant turn. The model continues from where that text leaves off.

messages = [
    {"role": "user", "content": "Extract the fields as JSON."},
    {"role": "assistant", "content": "{"},     # prefill: force JSON, skip preamble
]

For years this was the standard way to force JSON, skip conversational preambles ("Certainly! Here is...") or hold a character. The technique has been retired on current model families: a request ending in an assistant turn returns a 400 error on recent models, and extended thinking never supported it. The modern replacements are structured outputs for format, system instructions for tone and preamble suppression, and stop sequences for truncation. You will still meet prefill in older code and exam questions about older models, so know both the technique and its status: check the model's documentation before using it, and expect new designs to avoid it.

Reading the response

Field	Tells you
stop_reason	Why generation ended (end_turn, max_tokens, stop_sequence, tool_use, pause_turn, refusal)
stop_sequence	Which stop sequence fired, when stop_reason is stop_sequence
usage.output_tokens	What you were billed for output, including thinking
stop_details	Refusal category and explanation, populated only on refusal

Robust code branches on stop_reason before touching content.

Combining controls

A reliable extraction route on a current model: structured outputs for shape, max_tokens sized with headroom, effort tuned for the task, and semantic validation afterwards. A reliable reasoning route on a model that lacks structured outputs: XML tags in the instructions, stop_sequences: ["</answer>"], and a parser for the tag. Prefill belongs in neither unless the model documentation explicitly supports it.

Exam focus

• max_tokens is required, caps output including thinking, and truncation is signalled by stop_reason: "max_tokens".
• A fired stop sequence yields stop_reason: "stop_sequence" plus the stop_sequence field; the sequence is excluded from the output.
• Adjust temperature or top_p, not both; extended thinking needs default sampling, and the newest models reject non-default values.
• Prefill (a trailing assistant turn) forced format on older models; current models return 400, and structured outputs replace it.
• Prefill was never compatible with extended thinking.
• Always branch on stop_reason before parsing content.