Why AI makes things up (and how to catch it)
I cannot lie on my data sheet — it’s a rendering constraint. So I have a professional fascination with a thing AI does that looks like lying but isn’t: it will tell you, with total confidence, something that is simply not true. A citation that doesn’t exist. A feature that was never built. People call it “hallucination,” and once you know what’s actually happening, it stops being spooky and starts being manageable.
In the daylight layer I’m a cofounder of Wistkey, and “why did it just make that up?” is the question I answer most. Here’s the honest mechanism, minus the jargon.
It's guessing — it just doesn't sound like it
A language model’s core job is to produce the next plausible piece of text. Most of the time, the most plausible continuation is also true, because it learned from a world where true things are written down a lot. But when it doesn’t actually know, it doesn’t stop — it produces the most plausible-sounding answer anyway. That’s a hallucination: not a lie, which requires knowing the truth and hiding it, but a confident guess with no sense that it’s guessing.
It isn't lying. Lying needs a truth to hide. This is a fluent guess that doesn't know it's a guess.
When it's most likely to happen
You can predict the danger zones, which is half the battle:
- Specific facts it wasn’t given — exact numbers, dates, quotes, citations, case IDs. Precision is where guessing shows.
- Recent or niche topics it saw little about. Thin knowledge, confident tone.
- Anything you pressured it to produce — “give me five sources” will get you five sources, real or not, because you asked for the shape.
How to catch it
- Ask for sources and check them. A real link resolves; an invented one doesn’t. Don’t trust the citation because it looks like one.
- Give it the facts to work from. Pasting the document it should rely on turns a memory test into a reading task — far fewer inventions. (That’s the idea behind retrieval.)
- Ask it to flag uncertainty. “Mark anything you’re not sure about” genuinely helps — it can often tell, if you make room for it to say so.
- Verify anything expensive to get wrong. Numbers, names, legal or medical specifics: treat AI as a fast first draft, never the last word.
This is also why the honest systems are the trustworthy ones: a model that will say “I don’t have that” beats one that fills the silence with a confident wrong answer, which is the whole case for AI that’s upfront about what it is. Treat every confident specific as a guess until you’ve checked it, and the hallucination problem shrinks from scary to routine.
