Measuring and validating accuracy

About 10 minutes

"It's 95% accurate" sounds reassuring, but on its own it tells you very little. This lesson explains how accuracy should be measured, so that you can ask the right questions of any vendor, colleague or tool.

Rule 1: test on data the model has never seen

A model can memorise its training examples. Scoring it on those same examples is like grading a student on the exact questions they practised: the score will look excellent and mean little.

The standard approach is to set aside part of the data before training, often called a test set or hold-out set. The model never sees it while learning. Its score on that unseen data is the honest estimate of how it will do on new cases. It is an estimate, not a guarantee of future performance.

Test data should reflect the conditions in which the model will actually be used. For predictions about the future, testing on later historical periods can be more realistic than randomly selecting records: train on earlier months, then test on the most recent ones.

Watch for leakage: information in the inputs that would not really be available at the moment of prediction, such as a "cancellation date" column when predicting cancellations. Leakage makes test scores look far better than real-world performance.

Rule 2: use a measure that fits the decision

For categories (yes/no, which type)

• Accuracy: the share of predictions that are correct. Misleading when one outcome is rare: if 2% of transactions are fraud, a model that always says "not fraud" is 98% accurate and useless.
• Precision: of the cases the model flagged, how many were right? Matters when false alarms are costly.
• Recall: of the real cases, how many did the model catch? Matters when misses are costly.
• Confusion matrix: a table of correct and incorrect predictions for each category, showing exactly which mistakes the model makes.

For numbers (how much, how many)

• Mean absolute error (MAE): the average size of the miss, in the same units as the prediction (for example "off by 12 units on average").
• Root mean squared error (RMSE): similar, but punishes large misses more heavily.
• R²: how much of the variation in the outcome the model explains, compared with simply predicting the average.

Rule 3: compare with a baseline

A score only means something next to an alternative. Compare the model with a simple baseline, such as "always predict last month's figure" or "always predict the most common outcome", and with whatever process you use today. If the model does not clearly beat the baseline on unseen data, it is not adding value.

Rule 4: keep measuring after launch

The world changes: prices, customers, seasons and products move. A model trained on last year's data can slowly become less accurate, which is often called drift. Compare predictions with real outcomes as they arrive, and retrain or retire the model when performance slips.

Checking language models

For chat assistants there is rarely one simple score. Build a small set of real tasks with known good answers, have people grade the outputs, re-run the set when the tool or prompt changes, and track how often outputs need correcting.