A hallucination is an output that sounds plausible but is factually wrong or invented. The tricky part is not the error — it is that it arrives in the same confident tone as everything correct.
Invented citations, a statutory provision that does not exist, a number that fits a little too well: hallucinations are the most common reason AI projects lose trust inside companies.
Important to understand: this is not a defect an update will fix. It follows from how these models work.
Why models invent #
A language model predicts which sequence of words fits best. It holds no internal register of "I know this" versus "I don't". Where no reliable pattern exists for a question, a fluent answer still appears — just an invented one.
That is why hallucinations are most likely around precise individual facts: names, numbers, dates, case numbers, quotations. And least likely around structure, phrasing and summarising.
The more checkable a detail is, the more you need to check it. Summarising and rephrasing can largely be delegated — asserting facts cannot.
How to reduce the risk #
- Supply sources instead of querying memory. A model answering from an attached document invents far less than one answering from recall. That is exactly what RAG provides.
- Demand evidence. Ask in the prompt for the reference and the quote, and you get either something checkable — or a visible admission.
- Permit uncertainty. "Say explicitly when you are not sure" works remarkably well.
- Two models against each other. For critical claims, put the same question to a second model — divergence is an alarm signal.
What I never take unchecked #
Anything someone could later hold against me: legal references, numbers in proposals, quotations, people's names and roles, historical dates.
An example from this blog: for the AI labelling fact check I looked up every reference individually — paragraph, section, page. Not because I distrust the model, but because an invented provision in an article about legal certainty would be the most expensive error imaginable.
Common questions #
Why does AI invent sources?
Because language models predict sequences of words rather than looking things up in a database. A citation has a very typical pattern — which can be produced fluently without the source existing.
Can hallucinations be switched off entirely?
No. They can be reduced substantially: with supplied sources, with an instruction to flag uncertainty, and with human checking of verifiable facts.
Do newer models hallucinate less?
Broadly yes, especially when allowed to attach sources. The problem has not disappeared — and the more confident the tone, the harder it is to spot.
How do I recognise a hallucination?
By checkable details that fit too neatly: an exact figure with no source, a quote with no reference, a provision saying precisely what you hoped to hear. Always verify those passages.
Sources #
- Anthropic — Reduce hallucinations
- Regulation (EU) 2024/1689 (AI Act) — Art. 4: obligation of sufficient AI literacy for deployers
- Own practice: reference-by-reference verification in the AI labelling fact check, August 2026
How do you verify AI output?
Ask me — or ask Ada. She is my AI assistant, available around the clock, and will propose a slot with me if you need one. Free, no form.