Retrieval, Citations and Grounding

A model answers from what is in its context. Grounding means ensuring the facts an answer depends on are in that context, come from a trusted source and are traceable afterwards. Retrieval-augmented generation (RAG) supplies the facts; citations make the dependence visible; prompt design keeps the model from filling gaps with plausible invention. The exam tests the API features for documents and citations and the design patterns that make grounding reliable.

The retrieval pattern

1. Index a corpus into chunks (with embeddings, a keyword index or both).
2. At request time, retrieve the chunks most relevant to the query.
3. Place them in the prompt as documents, followed by the question.
4. Instruct the model to answer only from the documents and to say when they do not contain the answer.

The quality of step 2 bounds everything after it. Hybrid retrieval (semantic plus lexical) outperforms either alone on most corpora, and reranking the top candidates with a second pass improves precision. Contextual retrieval, where each chunk is prefixed with a short model-generated description of where it sits in its source document before indexing, reduces the failure in which a chunk is retrieved but unintelligible out of context.

Document blocks

The API has a first-class content type for source material. A document block carries a source (plain text, base64 PDF, a url, a file ID from the Files API, or custom content made of pre-chunked text blocks), an optional title, an optional context string that is shown to the model but never cited, and a citations setting.

messages = [{"role": "user", "content": [
    {"type": "document",
     "source": {"type": "text", "media_type": "text/plain", "data": policy_text},
     "title": "Refund policy v7",
     "context": "Internal policy, effective March",
     "citations": {"enabled": True}},
    {"type": "text", "text": "Can a customer get a refund after 45 days?"},
]}]

Place documents before the question. The Files API lets you upload a document once and reference it by ID across many requests, which saves bandwidth and keeps the request bodies small.

Citations

With citations enabled on every document block, the response arrives as a sequence of text blocks, some carrying a citations array. Each citation includes the cited_text, the document_index and document_title, and a location whose type depends on the source: char_location (start and end character indices) for plain text, page_location (page numbers) for PDFs, and content_block_location (block indices) for custom content. Your application can render these as footnotes, highlight the source passage, or verify that the cited text actually appears in the document. Cited text does not count towards output tokens.

Two constraints: citations must be enabled on all documents in a request or none, and citations cannot be combined with structured outputs.

For material that arrives through tools rather than in the prompt, the search_result content block plays the same role: a tool result can return blocks with a source, a title, content and citations enabled, so that a RAG step implemented as a tool still yields verifiable citations. Anthropic's web search and web fetch server tools return citations automatically.

Prompting for grounded answers

• Constrain the source. "Answer using only the documents provided. If they do not contain the answer, say so."
• Quote first. Ask for relevant quotes in <quotes> tags before the answer in <answer> tags. Extraction before synthesis measurably reduces invention and gives you something to check.
• Permit abstention. Models hallucinate most when forced to answer; make "not found in the documents" an acceptable output and test that it is used.
• Separate instructions from content. Documents go in tagged blocks; instructions stay in the system prompt. Content retrieved from the web or from users is untrusted and must not be able to change the instructions.
• Do not pad. Ten loosely relevant chunks dilute three relevant ones. Retrieve fewer, better chunks and let the model ask for more through a tool if needed.

Verifying grounding

Grounding is testable. Build an evaluation set with questions, the documents that should answer them and the expected answers, and grade on three axes: correctness, whether every claim is supported by a citation, and whether the model abstains when the answer is absent. Check cited spans programmatically against the source text; a citation that does not match is a bug in the pipeline, not a judgement call.

Choosing a mechanism

Situation	Mechanism
Few, known documents per request	Document blocks in the prompt with citations
Large corpus, per-query retrieval	Your retriever as a tool returning search_result blocks
Public web information	Web search and web fetch server tools
Same large file across many requests	Files API plus document block by file_id

Exam focus

• RAG quality is bounded by retrieval; hybrid search, reranking and contextual retrieval address the common failures.
• document blocks accept text, base64 PDF, URL, Files API IDs or custom content, with title, context (not cited) and citations.
• Citation locations are char_location for text, page_location for PDFs, content_block_location for custom content; cited text is not billed as output.
• Citations are all-or-nothing across documents and incompatible with structured outputs.
• search_result blocks give tool-returned content the same citation behaviour.
• Grounded prompting constrains the source, extracts quotes first and permits abstention; verify citations programmatically.