5.16 Anomaly detection

What this is and why it exists

Anomaly detection flags the rare and the strange without anybody having labelled examples of it, which is what makes it valuable — the interesting events in fraud, faults and monitoring are exactly the ones you have too few of to learn from directly. It is also the setting where evaluation is hardest, and where a detector that looks excellent by the usual numbers can be worthless. So the evaluation question comes first here, not last.

The vocabulary

• Anomaly — a point that does not look like the rest; not necessarily an error, and not necessarily interesting.
• Point anomaly — a single odd value; contextual — odd for its context; collective — a group that is odd together while each member looks fine.
• Robust z-score — distance from the median in units of a spread measure that outliers cannot inflate.
• Isolation Forest — a method scoring points by how few random splits it takes to separate them.
• One-class boundary method — learning a boundary around the normal data and flagging what falls outside.
• Reconstruction error — how badly a model that learned to rebuild normal data rebuilds this point.
• Contamination — the assumed share of anomalies, which many tools ask for.

The mental model

Three kinds of anomaly, and confusing them is why detectors miss the thing they were built for. A point anomaly is a value far from the rest — a transaction of ten lakh among transactions of a few thousand. A contextual anomaly is normal in general and wrong here: heavy electricity use is unremarkable in summer and strange at three in the morning in January, so the context has to be part of what the detector sees. A collective anomaly is a sequence in which no single value is unusual but the pattern is — a hundred small transfers in a minute, each entirely ordinary. Ask which kind you are looking for before choosing a method, because a point detector will never find a collective anomaly however well it is tuned.

The methods, in increasing order of what they assume and what they cost.

Statistical thresholds are the honest starting point. Flag values more than a few units of spread from the centre — using the median and a robust spread measure rather than the mean and standard deviation, because a handful of extreme values inflates the ordinary versions and hides the very things you are hunting. This is interpretable, instant, and correct far more often than its reputation suggests. Apply it per segment where the population is mixed, which handles many contextual cases for almost no work.

Isolation Forest rests on a neat inversion: instead of modelling what is normal, it measures how easily a point can be separated by random splits. Anomalies, being few and different, get isolated in a handful of splits; ordinary points need many. It handles many features, makes no distributional assumption, and is fast — which makes it the sensible default for tabular anomaly detection.

One-class boundary methods learn a surface around the normal region and flag what falls outside. They are more principled and much more sensitive to scaling and settings, and they scale poorly with data size for the same reason kernel machines do.

Reconstruction error applies where the data is high-dimensional and structured — images, signals, sequences. Train a model to compress and rebuild normal examples; it becomes good at normal and bad at everything else, so a large reconstruction error is a signal. It is the natural bridge to the deep-learning module, and it inherits that module's costs: more data, more compute, and less interpretability.

Then the evaluation problem, which is this topic's real content. Accuracy is meaningless — anomalies are perhaps one in a thousand, so a detector that flags nothing is 99.9 percent accurate. Precision and recall are the right pair, and even they need care, because the ground truth is usually incomplete: you know about the frauds that were caught, not the ones that were not, so a "false positive" may be a true positive nobody confirmed.

So decide the protocol before building the detector, and there are three honest options. Use a labelled historical period if one exists, accepting that its labels are a lower bound. Measure precision at a fixed budget — of the fifty cases we can investigate a day, how many are real — which matches how the output will actually be used and sidesteps the unknown denominator. Or inject known synthetic anomalies and measure how many are caught, remembering that you are then measuring detection of the kind you invented.

And the setting that quietly decides everything: the contamination rate, which several methods ask for. It is a statement about how much of your data you believe is anomalous, and getting it wrong shifts every score. Estimate it from the investigation capacity rather than guessing — if fifty cases a day can be looked at, that is the threshold you actually want.

What you should now be able to explain or do

Name the three kinds of anomaly and say which method families can and cannot find each. Explain why the median and a robust spread beat the mean and standard deviation here. Say what Isolation Forest measures and why it is a good default. Describe when reconstruction error is the right approach and what it costs. Explain why accuracy is meaningless and why even precision needs care. Choose one of the three evaluation protocols for a described situation. Say what the contamination setting means and how to choose it.

Check yourself