LLM Hallucination: Why It Happens and How to Reduce It
LLM Hallucination is for builders who need the term to survive contact with real products, tools, and failure modes. The goal is a practical mental model you can use in design review, not a glossary definition.
We will follow one concrete workflow, separate mechanism from product language, and end with a checkable practice. For nearby background, connect this with Classic RAG vs Graph RAG vs Agentic RAG and Structured LLM Outputs.
Hallucination Is Confident Text Without Grounding
An LLM predicts likely text from context and training, not verified facts from a database unless you connect one. Hallucination happens when the model produces a plausible answer that is unsupported, outdated, contradicted, or outside the evidence it actually has.
The running example is a docs assistant answering whether API endpoint /v2/refunds supports partial refunds. The correct behavior is not always answering; it is grounding the answer in the current docs or abstaining.
Quick reference
- Grounding means the answer is tied to supplied evidence.
- Plausibility is not the same as truth.
- RAG reduces some hallucinations but can introduce retrieval errors.
- Validation catches contract errors, not every factual error.
Remember this
A hallucination is a grounding failure: fluent text outruns available evidence.
Reduce Hallucination With Evidence Boundaries
The strongest controls happen before generation: retrieve the right documents, keep irrelevant context out, label sources, and ask the model to answer only from evidence. After generation, check citations, schema, dates, and claims that can be validated by tools.
For /v2/refunds, the system should retrieve the current endpoint page, preserve version metadata, and require the answer to cite the exact section. If the page is missing, the answer should say that instead of filling the gap.
Quick reference
- Retrieve narrow, versioned chunks with source metadata.
- Ask for citations to evidence actually present in the prompt.
- Use tools for live facts, calculations, and permissions.
- Use abstention when required evidence is absent.
Remember this
The best hallucination control is evidence selection plus an output contract that allows I cannot verify this.
Failure Story: RAG Retrieves the Wrong Version
Trigger: the vector search returns a deprecated /v1/refunds page because it is semantically similar. Symptom: the assistant claims partial refunds are unsupported. Root mechanism: retrieval optimized similarity, not version correctness.
Recovery is to filter by API version and expose the cited source in the answer. Prevention is to test retrieval separately from generation: given a query, does the system fetch the right document before the model writes anything?
Quick reference
- Evaluate retrieval hit rate before evaluating final answers.
- Filter by tenant, version, locale, and permission before similarity ranking.
- Show source titles and dates in logs.
- Add canary questions whose answers changed across versions.
Remember this
RAG can move hallucination from generation to retrieval; both stages need tests.
Practice: Create a No-Evidence Test
Write five questions your docs can answer and five plausible questions they cannot. Require responses to include answer, citations, and confidence.
Pass when answerable questions cite the right page and unanswerable questions abstain. Break it by removing the relevant document from retrieval; recover by returning not_enough_evidence instead of a guessed answer.
Quick reference
- Starter: use a small markdown folder as the document source.
- Expected success: no citation, no factual claim.
- Intentional break: remove the newest refund doc.
- Recovery: return an explicit uncertainty status.
Remember this
A hallucination defense passes only when the system refuses unsupported answers under test.
Key takeaway
Use the concept when it gives you a clearer boundary, measurement, or operating rule. Skip the label when it only makes the system sound more advanced.
Practice should leave behind an artifact: a table, eval set, trace, or checklist that another engineer can inspect. Pass when the artifact catches the intentional failure described above and gives a concrete recovery path.
Related Articles
Explore this topic