Hallucinations and How to Spot Them

A hallucination is output that is fluent, confident and wrong: a statistic that does not exist, a quotation nobody said, a regulation with the wrong clause number, a feature a product never had. It is the single most important concept in this domain because it is the failure mode that causes real workplace harm, and because the text that contains it looks exactly like text that is correct.

Why it happens

Claude generates text by predicting what is likely to come next given everything in the conversation. For most tasks that is exactly what you want; it is why drafts read naturally. But when a prompt asks for a specific fact Claude does not have, the most likely continuation is still a plausible-sounding fact. The model is not lying; it has no reliable way to distinguish "I know this" from "this is the kind of thing that would be true". Modern Claude models are much better at saying "I am not certain" than earlier ones, but the risk never reaches zero, and it rises sharply when the prompt demands precision the model cannot supply.

The high-risk zones

Hallucinations cluster in predictable places. Treat anything in these categories as unverified until you have checked it.

• Citations and references: paper titles, authors, URLs, case law, page numbers. Fabricated references are the classic example and still appear.
• Precise numbers: market sizes, percentages, dates, prices, version numbers. Round, plausible figures are a warning sign.
• Quotations: attributed quotes, especially from executives, politicians or historical figures.
• Niche or recent facts: anything about a small company, a local regulation, or events after the model's training data ends.
• Your own organisation: Claude knows nothing about your internal policies, people or products unless you supply them, yet it will happily describe them.
• Details inside long documents: when summarising, Claude may blend two sections or attribute a point to the wrong party.

Patterns that should trigger a check

Experienced users develop a feel for suspicious output. The following patterns are worth memorising.

1. Unprompted specificity. You asked a general question and received an exact figure, name or date you did not supply.
2. Suspiciously tidy evidence. Three studies that all support the point, with no counter-evidence, all neatly summarised.
3. No hedging where hedging is due. Confident claims about contested or fast-moving topics.
4. Internal inconsistency. A number in paragraph two that does not match the table in paragraph four.
5. Convenient alignment. The output agrees a little too neatly with the premise of your question, even when your premise was shaky.
6. References you cannot immediately find. If a search for the exact title returns nothing, assume it does not exist.

Scenario: the board paper

An operations director asks Claude to "add some industry benchmarks" to a draft board paper on warehouse automation. Claude inserts a paragraph stating that "according to a 2023 McKinsey study, 61% of mid-sized logistics firms reported payback within 18 months". The sentence is fluent and specific. The director searches for the study. It does not exist in that form; a similar report exists with different figures and a different scope. Had the paragraph reached the board, the director would have been defending an invented statistic. The fix at prompt stage would have been: "Suggest the kinds of benchmarks I should look for and where, but do not state figures; I will source them."

Reducing the risk before you read the answer

Good evaluation starts with the prompt.

• Supply the facts yourself and instruct Claude to use only the material provided.
• Ask Claude to say when it is unsure: "If you do not know, say so rather than guessing."
• Enable web search or use Research when the task depends on current or verifiable information, so claims are tied to sources you can open.
• Separate drafting from fact-finding. Let Claude structure the argument with placeholders such as [figure needed], then fill the placeholders from real sources.
• Ask for the source of each claim: "After each factual statement, note where it came from: the attached document, general knowledge, or an assumption."

That last instruction is powerful because it forces Claude to label its own confidence, and "general knowledge" or "assumption" tags tell you exactly where to look.

What hallucination is not

Not every error is a hallucination. A misread instruction, a formatting slip or an opinion you disagree with are different problems with different fixes (Domain 7 covers diagnosis). The term applies to fabricated factual content presented as true. Keeping the definition tight matters on the exam, where distractors often describe stylistic or formatting failures and label them hallucinations.

Exam focus

• A hallucination is fluent, confident, fabricated factual content; it is not a tone or formatting problem.
• Highest-risk content: citations, precise numbers, quotations, niche or recent facts, and anything about your own organisation that you did not supply.
• Warning patterns include unprompted specificity, suspiciously tidy evidence, missing hedges and internal inconsistency.
• Prompt-stage mitigations: supply the facts, restrict Claude to provided material, ask it to flag uncertainty, use placeholders, and label the source of each claim.
• Web search and Research reduce risk by tying claims to sources you can open, but they do not remove the need to check.