Prompt Iteration, Templates and Versioning

A prompt that works in a chat window is a prototype. A prompt that works on a thousand varied inputs, keeps working after a model update, and can be rolled back when it does not is an engineered artefact. The exam tests the workflow Anthropic recommends for getting from the first to the second, the role of templates and variables, and the versioning discipline that makes prompt changes as safe as code changes.

The iteration workflow

Anthropic's recommended sequence is deliberately test-first:

1. Define success criteria. Measurable and specific: "95% of tickets classified into the correct category", "no response exceeds 120 words", "every claim cites a document". Vague criteria ("good summaries") cannot drive iteration.
2. Build an evaluation set. Realistic inputs with expected outputs or grading rubrics, including edge cases and known failure modes. Start small (a few dozen cases) and grow it as failures are discovered.
3. Draft the prompt. Clear instructions, examples, structure, the techniques from the preceding lessons.
4. Run the evaluation and inspect failures. Not just the score: read the failing cases and categorise the causes.
5. Change one thing, re-run. Attribute improvements to specific changes.
6. Ship, monitor, feed production failures back into the set.

The Anthropic Console supports this loop with a Workbench for drafting, a prompt generator that produces a first draft from a task description, a prompt improver that applies best practices to an existing prompt, and an evaluation tab for running a prompt over a test set and comparing versions side by side. These are accelerators for the workflow, not substitutes for it.

Templates and variables

Production prompts separate the fixed instructions from the variable input. In the Console, templates use double-brace variables such as {{DOCUMENT}} and {{QUESTION}}. In code, the same idea is a function that assembles the request:

SYSTEM_V3 = open("prompts/support_system_v3.txt").read()

def build_messages(docs: list[str], question: str):
    doc_blocks = "\n".join(
        f'<document index="{i}">{d}</document>' for i, d in enumerate(docs, 1))
    return [{"role": "user", "content": f"<documents>\n{doc_blocks}\n</documents>\n\n{question}"}]

Templating rules that matter in practice: escape or delimit user-supplied values so they cannot close a tag or impersonate an instruction; keep the stable part of the template first so that caching applies; and never interpolate volatile values (timestamps, request IDs) into the cached prefix.

Versioning

Treat prompts as code:

• Store them in the repository, in files or constants, not in a database field edited by hand.
• Give each a version, and log the version with every request so that production behaviour can be traced to the prompt that produced it. The metadata field on a request is a convenient place for a tag, or log it alongside the request ID.
• Pin the model explicitly. A prompt is tuned against a model; "latest" aliases change underneath you. Model upgrades are prompt changes and go through the same evaluation.
• Review changes in pull requests with the evaluation results attached.
• Keep the previous version deployable for rollback.

Note that any change to the system prompt or tool definitions invalidates the prompt cache for every request; batch prompt changes rather than tweaking continuously, and expect a cost bump on deployment.

Iterating on failures, not feelings

When a prompt fails, resist the urge to add another sentence of instruction. Classify the failure first:

Failure pattern	Likely fix
Wrong format	Add an example or use structured outputs
Ignores a rule under some inputs	Move the rule earlier, make it positive, add an example that exercises it
Hallucinated detail	Ground with documents and ask for quotes; permit "not found"
Inconsistent across similar inputs	Lower temperature; tighten the definition the model is applying
Over-long or verbose	State the length explicitly; remove instructions that reward thoroughness
Fails only on a new model	Remove old-model workarounds (forced reasoning, repeated emphasis) and re-tune effort

Each fix is a hypothesis; the evaluation set is how you test it.

Regression protection

Once a prompt is in production, every change to the prompt, the model, the tools or the retrieval pipeline should run the evaluation set automatically, with a threshold that blocks deployment on regression. Prompts drift in quality for reasons unrelated to their text, such as a change in the documents retrieved or an increase in input length, so keep the set representative of current traffic.

Exam focus

• The workflow is criteria first, evaluation set second, prompt third; iterate against measured failures, one change at a time.
• Templates separate stable instructions from variable input; variables must be delimited so user content cannot impersonate instructions.
• Prompts are versioned in source control, logged per request, and reviewed with evaluation results.
• Model versions are pinned; a model upgrade is a prompt change and needs the same evaluation.
• Changing the system prompt or tools invalidates the prompt cache for all requests.
• Console tools (prompt generator, improver, evaluation) accelerate the loop but do not replace success criteria and test sets.