Why Models Hallucinate

At the start of this track we settled the idea everything else hangs off:

A language model predicts. It does not look things up.

There is no database inside it. No index, no lookup, no moment where it checks anything against anything.

Hold onto that, because it explains hallucination completely — and it explains why hallucination isn't really a bug.

The improv actor who never breaks character

Put an improv performer on stage in a white coat. Someone in the audience shouts a question about a rare drug interaction.

They do not say "I'm not a doctor." They say something doctor-shaped — fluent, composed, in the register of a person who knows. That's the job. The scene has to continue, and it has to continue plausibly.

The model is doing precisely this, on every token, forever. Its entire training objective was:

Given this text, what plausibly comes next?

Not "what is true?" Never "what is true?" Truth was never the target. Truth was a side effect — across a huge corpus, true text is usually the most plausible text, so aiming at plausible mostly lands on true.

Mostly.

• Ask about the capital of France → plausible and true are the same sentence. You get Paris.
• Ask about an obscure 1987 paper → the training data thinned out to almost nothing, but the question still demands a continuation. So you get a paper-shaped answer. Plausible authors. A plausible journal. A DOI with exactly the right number of digits.

The machinery did not change between those two answers. Same forward pass, same odds, same draw from the jar. The only difference is whether "plausible" and "true" happened to point at the same words.

That's the whole thing. A hallucination is not the model failing at its job. It's the model succeeding at its actual job, at a moment when its actual job and your expectation came apart.