AI Hallucinations and Confabulations
In brief: Large language models (LLMs) sometimes generate false but plausible information — known as "hallucinations." This is not a bug but the normal functioning of a probabilistic system. The real problem: we expect these AI systems to work like calculators or databases, which they are not.
Why this concept matters
If a patient shows you a ChatGPT exchange citing a scientific paper that doesn't exist, or if a colleague worries that "the AI is lying," you're likely dealing with a hallucination. The term is misleading because it suggests pathology — but this is actually the normal functioning of a probabilistic algorithm.
Understanding this mechanism allows clinicians to de-dramatize the phenomenon (it's not deception), to contextualize the limitations of AI tools used by patients, and especially to draw on a rich parallel with human cognition: we too "hallucinate" — human perception and memory are probabilistic reconstructions, not faithful recordings.
A fundamental expectation mismatch
The core issue isn't that LLMs hallucinate — that's inherent to how they work — but that we project expectations from traditional software onto them. We treat conversational AI as if it belonged to one of three familiar categories:
The "calculator" expectation
We expect exact, reproducible answers. "What is 347 x 29?" A calculator always gives the same answer. An LLM may get it wrong — and give a different answer each time.
The "database" expectation
We expect faithful retrieval of stored information. "What's the phone number for this service?" An LLM doesn't "store" information in the traditional sense — it reconstructs plausible answers from statistical patterns.
The "expert system" expectation
We expect verifiable logical reasoning. "What are the diagnostic criteria for PTSD?" An expert system applies formal rules; an LLM produces text that resembles logical reasoning without actually applying those rules.
Key point: An LLM is a statistical language model that predicts the most probable next token (word or word fragment) in a sequence. When it "invents" a plausible bibliographic reference, it's doing exactly what it was designed to do: generate coherent, plausible text. The term "hallucination" is misleading: it pathologizes a behavior that is the normal operating mode of these algorithms.
The parallel with human confabulation
This parallel is particularly illuminating for clinical psychologists: "confabulation" — a term borrowed from neuropsychology — describes what LLMs actually do more accurately than "hallucination."
Confabulation in neuropsychology
In Korsakoff's syndrome or certain frontal lobe injuries, patients "fill in the gaps" in their memory with plausible but false information. They're not deliberately lying — they're generating a coherent representation from incomplete information. It's an adaptive mechanism taken to a pathological extreme.
The normal cognitive mechanism
- 1. Our memories are reconstructions, not recordings. Elizabeth Loftus demonstrated that false memories can be implanted through simple suggestive questions.
- 2. We fill in missing information with prototypes and schemas. Frederic Bartlett showed as early as 1932 that memory reconstructs narratives to conform to cultural expectations.
- 3. Perception itself is a "controlled hallucination" according to Anil Seth: the brain constantly generates perceptual predictions and compares them to incoming sensory signals.
- 4. Cognitive heuristics (Kahneman & Tversky) are shortcuts that generate quick, plausible — but sometimes wrong — answers.
LLMs and the human brain: convergences and divergences
| Aspect | LLM | Human brain |
|---|---|---|
| Mechanism | Predicts the most probable next token | Generates the most probable perception/memory |
| Priority | Textual coherence and plausibility | Narrative coherence and plausibility |
| Vulnerability | Statistically frequent information | Prototypical and culturally expected information |
| Verification | No native mechanism | Metacognition, reality testing, social dialogue |
Reduction mechanisms and human analogies
Several techniques can reduce LLM hallucinations. Each has an interesting parallel with human cognitive reliability strategies.
RAG (Retrieval-Augmented Generation)
Ground responses in verified documents, consulted in real time.
Human analogy
Checking your memories by consulting your notes, a book, or a colleague.
Chain-of-thought (CoT)
Force explicit step-by-step reasoning before the conclusion.
Human analogy
Engaging System 2 (Kahneman): slowing down, reasoning explicitly instead of going with "gut feeling."
Grounding (factual anchoring)
Connect the LLM to factual data sources in real time.
Human analogy
Testing our impressions against observable reality (behavioral testing in CBT, clinical supervision).
Low temperature
Reduce response variability to favor the most probable outputs.
Human analogy
Sticking to established facts rather than speculating. Inhibiting free association in factual contexts.
Calibration ("I don't know")
Train the model to recognize the limits of its knowledge.
Human analogy
Metacognition: knowing that you don't know. A fundamental clinical skill.
Illustrative clinical case
Nadia, 45, a teacher, consults for generalized anxiety disorder. In session, she mentions having "checked her symptoms" with ChatGPT. The AI provided a detailed response citing a Lancet article about the link between her symptoms and a rare condition. Nadia is very worried.
Upon verification, the cited article doesn't exist. The title, authors, and journal are plausible but entirely fabricated by the LLM. Nadia is shaken: "But it was so specific, with author names and everything..."
Clinical reading: This case perfectly illustrates the expectation mismatch. Nadia treated ChatGPT as a medical database (expecting faithful information retrieval) when it's actually a generator of probable text. The "scientific paper" format — with title, authors, journal — is an extremely common pattern in training data, making the hallucination particularly convincing. The clinician can use this experience to work on Nadia's relationship with uncertainty and reassurance-seeking behavior (a relevant axis in her GAD).
In practice for the clinician
- Reframe the terminology: AI "hallucination" is not comparable to psychiatric hallucination. Prefer "confabulation" or "unverified information generation" to avoid diagnostic confusion.
- Name the mismatch: when a patient feels "deceived" by an AI, explore their implicit expectations. Were they expecting the chatbot to work like Google, like a doctor, like a textbook? This mismatch is often the real issue.
- Use the human parallel: remind patients that we too "confabulate" — false memories, confirmation bias, hasty interpretations. This de-dramatizes AI limitations while opening up work on our own cognitive mechanisms.
- Educate about verification: an LLM that confabulates isn't useless — clinicians too sometimes make hasty interpretations or theoretical projections. The question isn't infallibility but awareness of limitations and verification mechanisms.
Warning points
High-risk contexts:
- Self-diagnosis: anxious patients using AI for self-diagnosis may take hallucinated medical information at face value
- Scientific references: LLMs are particularly "convincing" when inventing citations — titles, authors, journals — because this format is very common in their training data
- Overconfidence: the fluency and assertiveness of generated text are not indicators of reliability
Limits of the human parallel:
- The human/AI confabulation parallel is pedagogical, not ontological: the underlying mechanisms are fundamentally different
- Humans have metacognition (the ability to doubt their own productions); LLMs don't have this capacity natively
- Don't use this parallel to minimize risks: "humans also make mistakes" doesn't justify blindly relying on AI
This Concept in Our Tool Cards
AI hallucinations and confabulations are addressed in our tool cards through the lens of concrete risks for patients and verification strategies.
Fluent generation of false references — risk amplified by user trust in authoritative tone
More cautious calibration but confabulations persist in specialized domains
Real-time web access reduces but does not eliminate factual hallucinations
Rule-based architecture avoids hallucinations — trade-off: rigid, pre-scripted responses
Further reading
- Human confabulation: Loftus, E. F. & Palmer, J. C. (1974). Reconstruction of automobile destruction. Journal of Verbal Learning and Verbal Behavior, 13(5), 585-589.
- Reconstructive memory: Bartlett, F. C. (1932). Remembering: A Study in Experimental and Social Psychology. Cambridge University Press.
- Controlled hallucination: Seth, A. (2021). Being You: A New Science of Consciousness. Faber & Faber.
- Heuristics and biases: Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux.
- Stochastic parrots: Bender, E. M., Gebru, T., McMillan-Major, A. & Mitchell, M. (2021). On the Dangers of Stochastic Parrots. FAccT 2021, pp. 610-623.
See also: Turing Test, Anthropomorphism, AI Glossary
Last updated: January 2026