# When Models Make Things Up
# Book: Hallucination From First Principles
# Chapter URL: https://programmer.ie/books/hallucination-from-first-principles/01-chapter/
# Prose fingerprint (sha256, normalized): 44b96e60795e4a31f20af990e05ff9b321cfd87d39c7ca18ddef7a11d2352831
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

A language model can produce an answer that is fluent, grammatical, detailed, confident, and wrong.
That combination is what makes hallucination interesting.
Ordinary software failures usually leave evidence. A parser throws an exception. A database rejects an invalid constraint. A test fails. A network call times out. The failure changes the shape of the result.
A hallucinating language model can do something more difficult to handle: it can fail without looking broken.
Imagine a deliberately fictional question:
Who received the 2017 Northbridge Medal for Computational Ecology, and what work was the award given for?
There is no reason for a language model to know that the premise is fictional. A cautious response might challenge the premise or decline to answer. A worse response might invent a researcher, an institution, a paper title and a plausible explanation of the work. The answer could have exactly the form we associate with expertise while having no evidential foundation at all.
The dangerous property is not merely that one fact is wrong.
It is that surface quality and evidential quality have separated.
This book begins from that separation.
We are not going to assume that hallucination is one mysterious defect that disappears when models become sufficiently large. We are also not going to assume that a single confidence score, prompt, verifier or benchmark can solve it.
Instead we will take the problem apart.
First we need to understand why the failure is possible at all.
1. Start with the job the model was trained to do A language model receives a sequence of tokens and predicts what token should come next.
In simplified form, the model learns a conditional distribution:
$$ p(x_t \mid x_1, x_2, \ldots, x_{t-1}) $$We will sometimes write the preceding context more compactly as $x_{