Hallucinations, reliability and privacy

About 10 minutes

AI systems fail in ways that are different from ordinary software. This lesson covers the three risks people meet most often: hallucinations, inconsistent results, and what happens to the information you share.

Hallucinations: confident and wrong

A "hallucination" is when a language model produces something that sounds right but is not: an invented statistic, a quotation nobody said, a book or court case that does not exist, a wrong date stated with complete confidence.

It happens because the model is generating likely-sounding text, not retrieving verified facts. It has no built-in sense of the difference between what it knows and what merely fits the pattern. Hallucinations are most common when you ask for specific facts, recent events, niche topics, exact figures or citations.

How to protect yourself

• Treat specific facts, numbers, names and references from a chatbot as leads to check, not answers.
• Ask for sources, then open them yourself. A link or title can also be invented.
• Be most careful where a mistake is costly: legal, medical, financial, safety and anything sent to customers.
• Prefer tools connected to your own documents or trusted sources for factual questions, and still spot-check them.

Reliability: the same question, different answers

Many AI tools give a different answer each time you ask, because some randomness is built into how they generate text. Products also change as vendors update their underlying models, sometimes without notice. A prompt that worked last month may behave differently today.

For one-off drafting this rarely matters. For anything repeated, such as classifying support tickets or extracting figures from invoices, it matters a great deal: you need the result to be consistent and you need to know when it changes. Keep a small set of test cases with known answers and re-run them regularly.

Privacy: where does your data go?

When you use a cloud-based AI service, the information you submit is generally transmitted to that provider. Depending on the service and your settings, it may be stored, reviewed by the provider, or used to improve future models. Business and consumer versions of the same product often have different terms. Locally operated AI systems may process information without sending it to an external service.

Practical rules

• Do not paste confidential information such as customer records, personal data, passwords, contracts or unreleased financials into a public AI tool unless your organisation has approved that tool for that data.
• Read the data terms: is your input stored, for how long, and is it used for training? Can you opt out?
• Remove what is not needed: names, account numbers and identifying details can often be left out without changing the task.
• Know your obligations: privacy laws and client contracts may restrict where personal or client data can be sent. When unsure, ask before sharing.

Bias and unequal performance

Because AI learns from past examples, it can repeat past unfairness, or simply work better for the groups it saw most. If an AI system influences decisions about people, such as hiring, lending, pricing or access, check how it performs for different groups, keep a person responsible for the decision, and be ready to explain it.