Hallucination
hallucination
Hallucination is when a model confidently asserts something false. It is not a glitch or a bug; it is the natural output of a system trained to produce plausible text. The model has no internal flag distinguishing "I know this" from "this seems likely given the context."
Language models do not look up facts. They generate text that is statistically consistent with their training data and the prompt in front of them. When the answer to a question is well-represented in training and the question is phrased clearly, the output tends to be correct. When the answer is rare, contradictory, or absent from training, the model produces something that reads like an answer anyway. That is hallucination.
The defining feature of hallucination is not that it is wrong, but that it is wrong in a confident voice. Models do not naturally say "I don't know." They say "the result is X" or "the citation is Y." A model trained to refuse uncertain answers feels less helpful, so model providers tune the line carefully and never quite eliminate the behavior.
Defenses include: retrieval augmented generation (give the model the source documents and ask it to quote), tool use (let the model call a calculator, a search engine, a code interpreter), and explicit prompting for uncertainty. None of these are perfect. The /cannot section of this site demonstrates hallucination on purpose, with the same prompt rerun several times to show that the wrong answer is not even consistent across runs. That inconsistency is itself a tell.
This site uses the word "hallucination" because it is widely understood, but "confabulation" is a more accurate term: the behavior is closer to a person making up a memory than a person seeing things.