How to Spot When AI Has Drifted

You are the last check

There is no alert. Nothing turns amber when your AI assistant stops following your third rule, and nothing warns you that paragraph two is now built on an invention from an hour ago. The only detector in this system is you, reading, before you forward the thing to somebody who will assume you checked it.

The good news is that drift leaves fingerprints. Once you know the five, scanning for them takes about a minute.

Tell one: confident specifics you never supplied

This is the loudest signal, and the easiest to miss because specifics feel like quality.

Watch for numbers, dates, percentages, names, job titles and study citations that you cannot immediately trace to something you provided. Precision arriving from nowhere is not rigour. It is the shape rigour makes, produced because a specific number reads better than a vague one.

Round numbers deserve a second look too. So do phrases like "industry standard," "research consistently shows" and "most organisations find." Those are generic authority, and generic authority almost never has a source underneath it.

The most quoted worry in our learner research was AI giving false facts and fake sources in a tone that sounds completely confident. Confidence is not a dial the assistant turns up when it is sure. It is the default setting.

Tell two: the tone or format changed

You asked for plain sentences and paragraph three has grown headers. You set a brisk register and the last output has gone corporate. A generic consultancy voice has crept in where a colleague's voice used to be.

Format drift is worth taking seriously not because layout matters much, but because it is the visible edge of something invisible. If the shape of the output has slipped away from your instruction, the substance probably has too, and substance is far harder to eyeball.

Tell three: contradictions with earlier answers

Long sessions produce internal contradictions with a completely straight face. The scope was three regions in message ten and four regions in message fifty. A caveat that was firm at the start has quietly become a footnote.

These are hard to catch by reading forwards, because each answer is locally consistent. They surface when you compare, and comparing is a job you can hand back.

Prompt you can copy: the contradiction sweep

Review everything you have told me in this conversation.
List every place where a later answer contradicts, softens or quietly
changes an earlier one. For each, quote both versions and say which one
is supported by the material I supplied.
Do not resolve them by picking whichever sounds better. If neither is
supported by my material, say so.

Tell four: the hedging disappeared

This one is subtle and it is the one I would train yourself hardest on.

Early in a session the assistant says "the report suggests" and "this is not stated directly." Later, the same idea comes back as flat fact. Nothing new was learned in between. What happened is that the tentative version got restated, then restated again, and each pass sanded off a qualifier until the uncertainty was gone.

An idea that has become more certain without new evidence has not become more true. It has just been repeated. Treat vanished hedging as an alarm, not as progress.

Tell five: fluent formatting standing in for evidence

Tidy headers, confident bullets, a neat table with three columns. It reads like something that was checked. Nothing about producing a well laid out table requires the contents to be right, and a fabricated figure in a bordered cell looks more official than the same figure in a sentence.

Structure is a presentation choice. It carries exactly zero information about sourcing.

In plain English

Hedging:: Qualifying language such as suggests or appears, which signals the claim is not certain.
Confidence laundering:: A tentative claim becoming flat fact through repetition, with no new evidence added.
Provenance:: Where a specific claim came from: your source, the model's general knowledge, or its own earlier answer.
Verification pass:: A deliberate check of an answer against its source, run before the work goes anywhere near a colleague.

Make it show its working

Suspicion is not a method. When a tell fires, ask for provenance sentence by sentence.

Prompt you can copy: where did this come from

Take your last answer sentence by sentence. For each one, tell me:

Where it came from: my source material, your general knowledge, or
your own earlier answer in this conversation.
If it came from my source, the exact quote.
Your honest confidence, and what would change your mind.
Do not defend the answer and do not rewrite it. Just account for it.

The phrase "do not defend the answer" earns its place. Ask a question that sounds like a challenge and you often get a persuasive restatement of the same claim. Ask for an accounting and you get an accounting.

❌ Weak prompt

Prompt

Are you sure about that 34 percent figure?

Output

Yes, that figure is consistent with the analysis. The 34 percent reflects the proportion identified in the review and aligns with the broader pattern in the data. I am confident it accurately represents the finding.

You asked for reassurance and received it, at length. Not one word of that response is a quote, a page, or anything you could check. The figure is now more confident and no better sourced.

✅ Good prompt

Prompt

Do not defend the 34 percent figure. Tell me only this: which document, which page, and the exact sentence it came from. If it came from your own earlier answer rather than my source, say so plainly.

Output

It came from my own earlier answer in this conversation, in the summary I wrote at your third question. I cannot locate 34 percent in the material you supplied. The source states only that 'a substantial minority of respondents reported delays', with no figure attached.

One question, one honest answer, one invented statistic caught before it reached a slide. The specific instruction not to defend the claim is what made the difference.

The one-minute scan

Before anything leaves your hands, run five questions over it.

Your pre-send scan


Numbers and names: can I trace every one to something I supplied?
Rules: does this obey the constraints I set at the start?
Consistency: does anything here contradict what it told me earlier?
Certainty: has anything gone from hedged to flat without new evidence?
Authority: is anything here convincing me purely by looking tidy?
Any yes on 3, 4 or 5, or any no on 1 or 2, means run a verification pass
before this goes anywhere.

Scan hardest on the parts you liked most. The sentence that made you think oh, that is a good point is the one you are least likely to check, and the one a colleague is most likely to quote back at you.

Checkpoint

Drift shows up as unsourced specifics, changed tone or format, contradictions with earlier answers, hedging that quietly vanished, and tidy formatting doing the work of evidence. Ask for provenance sentence by sentence, and never treat fluency as a check.

📝 Quiz

Question 1 of 4

Which of these is the loudest early warning that an answer has drifted?