Lesson 4.3
When It Still Hallucinates
Grounding cuts hallucination, it does not erase it. The model can misread a passage, blend it with memory, or answer when the context is silent.
The open book is still read by a careless reader.
Handing the student the textbook helps a lot. It does not turn a careless reader into a careful one. They can still skim the wrong line, mix a half-remembered fact into the answer, or fill a gap with a confident guess when the page simply does not cover the question.
Grounding (lesson 4.1) is the same kind of help. It pulls the model's answers much closer to the sources and cuts the rate of made-up facts sharply. But the model is still doing the reading, and it can still slip. This lesson is about the slips that survive grounding, so you know what you are still on the hook to catch.
Grounding lowers the odds of a made-up answer. It does not lower them to zero. Treat it as a strong guardrail, not a guarantee.
Three ways a grounded answer goes wrong.
The failures cluster into three shapes. Naming them is half the battle, because each one has a different fix.
The silent context
The passages do not contain the answer, but the model answers anyway, filling the gap with a plausible guess instead of saying it does not know.
The misread
The answer is in the context, but the model pulls it from the wrong passage or scrambles a detail, so the citation points somewhere that does not back the claim.
Memory wins
The context says one thing, the model "knows" another from training, and it trusts its memory over the page in front of it, overriding the source.
That third one is the sneakiest. If the source contradicts something common, like a company that renamed a product or a fact that changed last year, the model's training nudges it back toward the old, more familiar answer, even though the fresh source is the one you want it to use.
Faithful is not the same as correct.
Two different questions hide inside "is this answer good?" The first: does the answer match the passages it was given (the precise term is faithfulness)? The second: is the answer true about the world (correctness)? They come apart in both directions.
Faithful but wrong
The source itself was outdated or mistaken. The model reported it accurately, so it was faithful, but the answer is false. The fix lives in retrieval, not generation.
Correct but unfaithful
The model happened to give the right answer from memory, with a citation that does not actually support it. Right today, but you cannot trust the process tomorrow.
The generation step can only be held to faithfulness: did it stick to the sources it was handed? Correctness also depends on whether the right sources were retrieved in the first place. Keeping the two apart tells you which part of the pipeline to go fix.
How to push the slips down.
None of these fully closes the gap, but stacked together they move the needle a lot.
- Tell it to abstain. Make "I do not know" an allowed, encouraged answer when the context is silent. A model that is permitted to decline guesses less.
- Check citations against the context. For each cited claim, confirm the cited passage actually contains it. A claim whose citation does not back it is the clearest signal of an unfaithful answer.
- Fix retrieval when the source is wrong. If the answer is faithful but false, the bad passage got retrieved. That is a retrieval problem (Unit 2), not a generation one.
Spot the unfaithful answer
The context handed to the model is a single passage:
[1] pricing.md: "The Pro plan costs 20 dollars per month and includes 5 seats."
Question: How much is the Pro plan, and how many seats?
Answer A: "The Pro plan is 20 dollars per month and includes 5 seats. [source: 1]"
Answer B: "The Pro plan is 20 dollars per month and includes unlimited seats. [source: 1]"
Which answer is faithful, and why? Answer A is faithful: every claim, the price and the seat count, is supported by passage 1, and the citation checks out. Answer B is unfaithful: the price is right, but "unlimited seats" appears nowhere in the source. It cites passage 1, yet passage 1 says 5 seats. The citation does not back the claim, which is exactly the misread (or memory-wins) failure. Notice that B is not even wrong-looking on its face; you only catch it by checking the citation against the context.
What grounding is not
A guarantee of truth
A citation says "I read this here," not "this is true." If the source was wrong or the model misread it, a confident citation sits right next to a bad answer.
A reason to stop checking
Grounding makes answers auditable; it does not do the audit. The cite-check, the abstain rule, and a retrieval that surfaces the right source still earn their keep.
The honest claim is not "RAG cannot hallucinate." It is "RAG hallucinates far less, and when it does, the citation gives you a place to catch it."
Key takeaways
Grounded answers still fail in three shapes: the context is silent so the model guesses, it misreads or misattributes a passage, or it overrides the source with memorized belief.
Faithful (matches the sources) and correct (true about the world) are not the same; an answer can be one without the other, and that tells you which part of the pipeline to fix.
Push the slips down by letting the model abstain when the context is silent and by checking each citation against the passage it points to; grounding is a guardrail, not a guarantee.