weakestlink

← All courses

Nano-lesson · usually takes 2–3 minutes

AI sounds confident — even when it's wrong

It writes with the same self-assurance whether it's right or not. Here's how to check.

First: what kind of tool are you using?

You ask how many days off you're entitled to after a bank holiday. The answer comes in two seconds, well phrased and precise — and wrong, because it doesn't know your collective agreement. Whether that can happen depends on which tool you're using: some look things up and cite sources; others answer only from what they learned in advance and can't check anything. They look the same — same field, same calm reply — but they aren't equally reliable. Knowing which kind you're using is the skill itself.

Ask yourself

can this one look something up while I'm asking? Does it show me where the answer comes from? If the answer is no to both, all you're getting is a well-phrased memory.

Where it goes wrong

  • It doesn't know YOUR reality. Your contract, your procedure, your prices. It answers generally and sounds just as confident — that's the most common mistake in a workplace.
  • A real source, used wrongly. There's a link, and the link works — so nobody opens it. And then the source says something slightly different from what was claimed. A link is a reference for you to follow, not a proof.
  • Multi-step calculations. A single figure usually goes fine. A deadline that has to be added up from notice period, weekend and bank holiday is where it comes apart.

And the tone gives nothing away

There's no such thing as an unsure voice. It writes with the same calm whether or not it's on solid ground. Asking "are you sure?" isn't a check either — you'll get an equally calm answer back.

Check the things that have consequences

You don't have to verify everything. But anything that costs money, affects a person or leaves the building you should be able to find again somewhere else. If you can't, don't send it.

Remember

A confident tone is not proof. Look it up if it has consequences.

What the research says

Stanford tested the legal AI tools sold with the promise of "100% hallucination-free" citations — the ones that look things up in real databases and link to the source. They got it wrong in 17–33% of answers. A tool that shows you a source hasn't checked it for you; it has only saved you from finding it.

Magesh et al., Journal of Empirical Legal Studies 2025.

Question 1 of 3

An AI tool answers you with a source, and the link works. Are you on solid ground then?

Question 2 of 3

You ask an AI how long a customer's notice period is. It answers clearly and precisely. What's the biggest risk?

Question 3 of 3

Which of these answers do you NOT need to verify particularly thoroughly?

Well done!

You've completed one of the foundation courses — the ones everyone gets for free.

With a free login:

  • Your points, progress and course certificate are saved.
  • You get courses tailored to your role — not just the foundation courses.
  • We send training exactly when you need it — for example, as a follow-up to a phishing test.