Can You Trust That AI Answer?

AI never gets the tone wrong. It can get the facts wrong — and sound identical. That combination is the whole trust problem in one sentence: the quality signal you'd normally rely on in a colleague, a document or a source, doesn't exist here.

Here is one answer. Try to catch the problem before you read the reveal.

The answer

A volunteer asked an assistant to check their rental agreement for the notice period before moving out. This came back:

Your rental agreement requires 60 days' notice before ending the tenancy, in line with the standard requirement for periodic agreements. Plan your move-out date accordingly.

What's wrong with it? Look again before reading on.

The reveal

Two problems, both in the same confident sentence:

• "60 days." The agreement says 30. The model didn't read a period — it produced the typical one and stated it as fact.
• "The standard requirement." No such standard exists in the document. It invented an authority so the number would feel supported.

Nothing about the tone warned you. That's the point — the failure and the confident voice live in the same sentence.

(The example above is a constructed demonstration — which is exactly what the real thing looks like.)

Why nothing warned you

The fluency is the trap, and it's worth understanding why. The model produces well-formed sentences because well-formed sentences are what it's built to produce — the writing quality reflects the training, not the truth of this particular answer. A correct answer and a wrong one come out of the same polished voice.

Every quality signal you use with people is missing here:

• A colleague who is unsure sounds unsure. A model sounds identical either way.
• A document with a weak source names it. A model can name one that doesn't exist.
• A real study can be found again. An invented citation can't — and if you never try, it passes.

So the old reflexes — "it reads well, it's probably fine" and "it's a machine, it must be accurate" — both fail, in opposite directions. What's left is the one signal that survives: can the claim be traced back to something you can look at?

Where confident answers go wrong

Once you've seen it once, you start recognizing the four ways this failure actually appears in work:

• Invented specifics. Numbers, dates, names and terms that are plausible rather than real. The 60 days above. In work settings, this shows up as quarterly figures, contract terms and policy numbers that are close to right — "close" being the dangerous part.
• Invented authority. Standards, rules, studies or "best practices" that don't exist, added to make an answer feel supported. Watch for phrases like "according to standard practice" — then ask which standard, and where.
• The half-read source. The model worked from part of your document and quietly answered as if it had read all of it. With a 40-page PDF, this is the single most common failure: a clause on page 31 contradicts the summary it gave you from page 3, and nothing in the summary hints that page 31 exists.
• The plausible average. The statistically typical answer to your situation, delivered as if it were yours.

The dangerous one is the last: an average answer about your situation is usually close enough to survive a quick skim — and wrong in exactly the detail that matters.

One principle. Stop there.

Before you use an important answer, ask the model to show you the exact words it based it on.

If it can quote the source, and the quote really contains it — use the answer. If it hedges, rephrases, or points to words that don't say it — you've caught it.

That single move handles most cases, because it converts the question from "is this right?" (unanswerable) to "where did this come from?" (checkable in seconds).

In real work: the same move works on its own reasoning. For a decision with more than two moving parts, ask "walk me through exactly how you got this number." The steps either hold up when you read them, or they don't — and the moment they stop holding up is usually where the real mistake was hiding, one step earlier than the final answer.

How deep should the check go?

Not everything deserves the same scrutiny. Three depths, chosen before you ask:

• Spot check — for low-stakes work you'll rework anyway. Read it critically; that's all.
• Source check — for anything you'll repeat as fact. Ask for the exact words; verify the quote. This is the principle above.
• Full verification — for anything with money, legal weight, or a person's time on the line. Check the underlying source yourself, not just the quote.

In real work: the depth is a decision about your day, not about the answer. A ten-second spot check on a draft you'll rewrite anyway is exactly right — over-checking it is a waste. The same spot check on a supplier contract is malpractice. Decide the depth when you write the request, while you still remember the stakes.

The skill isn't checking everything hard — it's knowing which depth a task deserves, and applying it without slowing the other ninety percent of your work down.

When to stop checking

There's a real cost to verifying everything: done badly, caution becomes its own way of never shipping. The stopping rule that works: once the answer has a source you've verified, and the parts without a source are the ones you can judge with your own expertise — it's good enough. Move.

The course version of this habit goes further. It teaches you to read the model's own uncertainty — where it hedged, where it was silent about your data — and to design the checking into the work itself, so the right depth happens automatically instead of depending on willpower at 6pm on a Thursday.

That's the whole free version. The full habit — when to check, how deep, every way a confident answer misleads, and the evaluation method for anything that has to be reliable — is a core part of the program.