Hallucination

We have a name for a certain kind of person at a hackathon: CBR. Confident But Wrong.

Directions delivered like gospel. The venue closed two years ago. The API key goes in .env, they swear — except your repo uses Varlock. Same calm voice either way.

That is an AI hallucination.

The model is not looking things up the way you would Google them. It is predicting the next most plausible piece of text, one scrap at a time. When it has the answer, that confidence is earned. When it does not, it often will not stop and say so. It generates something that looks like a right answer. Plausible. Detailed. Specific. Wrong.

That is what makes it sneaky. A hallucination does not look like a mistake. It looks like an answer — same tone as everything true it has ever told you. The fake file path sits right next to the real one wearing the same outfit.

Here is the moment it clicked for me. I asked an agent for the exact flag to turn on Convex with Better Auth in this stack. It gave me one that looked perfect. Right shape, right naming, the kind of thing the CLI would accept. The builder greys that combination out. I had never written that flag anywhere. The model invented a receipt that sounded like ours.

Lawyers have cited court cases that never existed. Same trick. Higher stakes.

So the fix is boring and it works: verify anything that matters.

• File paths — open the folder.
• CLI flags — paste into the builder or run with --help.
• "I already ran the migration" — check the database.
• A citation, a version number, a "this is deprecated" — follow the link.

If a wrong answer would cost you demo points or production sleep, check it yourself.

None of this means the tool is broken. It means you treat it like the brilliant, fast, slightly overconfident teammate it is. You would not ship an intern's research without a glance. Same energy here.

RAG gives it a book to cheat from. Tool calling lets it read the repo instead of guessing. Evals catch when the cheating stopped working. You still own the last look.

CBR. Confident But Wrong. Now you will see it in the chat and in the room.