How to Verify What an AI Tells You

Confident is not the same as correct

An AI assistant will never say it has a headache, never hedge because it is tired, and never trail off unsure. It answers every question in the same steady, well organised voice, whether the answer is solid, shaky or entirely invented. We are used to reading confidence as a signal of competence, because in people it usually is. Here it is just the house style.

The good news is that catching the errors does not require expertise. It requires a habit that takes about ninety seconds.

Why the fake sources look so good

This is the part that surprises people. If a model is going to invent something, why does it invent a reference with a plausible journal, a sensible year, page numbers and an author who genuinely works in that field?

Because formatting is the easy part. These systems are built to produce text that fits the pattern of the text around it. A citation is one of the most regular patterns in all of written English, so producing a flawless looking one is trivial. Whether the paper exists is a completely separate question that the pattern does not touch.

This is why the tidiness of a reference tells you nothing at all. Lawyers in several countries have been sanctioned for filing documents containing case citations that looked entirely ordinary and referred to cases that had never existed. They were not careless people. They were people who applied the normal test, which is does this look right, to a system that is exceptionally good at looking right.

The four step check

One: ask for sources up front. Do it in the original prompt, not afterwards. Ask for links, and add that it should tell you when it does not have one.

Two: follow at least one link. Not all of them, unless the stakes are high. One. Open it, and look for the specific claim, not just a page on roughly the right topic. Broken links, redirects to a homepage, and pages that turn out to say something slightly different are all extremely common.

Three: check a fact you already know. Ask about the part of the subject where you are the expert. If it gets your own field subtly wrong, and it often will, you have just calibrated how much to trust the rest.

Four: be most suspicious when the answer is exactly what you wanted. This is the one that saves careers. When an answer confirms the argument you were already making, your instinct to check disappears at precisely the moment you need it most.

Better prompts get more honest answers

You cannot make a model reliable, but you can make it easier to catch out.

Answer the question below. Then, before anything else:

List every factual claim you made that a reader could check.
Give a working link for each one, with the publisher and the date.
Mark anything you cannot source as UNVERIFIED and explain why.
Tell me which parts of your answer you are least confident about, and why.
Do not fill gaps with plausible-sounding detail. Missing is better than invented.
QUESTION:
[your question]

You have just given me an answer I agree with, which is exactly when I should be
careful. Now do the opposite job.

Make the strongest honest case against what you just told me.
List what would have to be true for your original answer to be wrong.
Tell me which specific facts I should check first to settle it.
Be genuinely critical. Do not soften it to keep me happy.

Before I rely on you for [topic], I want to calibrate.
Explain [a thing I already know well] in detail, including specifics, dates and
any exceptions.
I will check this against what I know. Be precise rather than general, and say
plainly where your knowledge may be out of date or incomplete.

Where errors cluster

Some answers deserve much more checking than others. Numbers, dates and direct quotes are the classic weak points, because a near miss reads exactly like a hit. Anything recent is risky, since a model's knowledge has a cutoff and it may not know what it does not know. Small organisations, local details and niche technical questions are thinner ground than famous subjects. And anything with real consequences (legal, medical, financial, or a claim you are about to put in front of a client) belongs in a different category entirely: verify it or do not use it.

Meanwhile, brainstorming, rewriting your own text, structuring an argument and explaining a concept you can sanity check yourself are all low risk. Spend your ninety seconds where it matters.