10.8 Monitoring, drift and retraining
You find out the model degraded before the user tells you.
Monitoring answers the question production asks forever: is the model still good? Service metrics watch uptime, drift detection watches the data, proxy metrics stand in while true labels are delayed. It sits late in the module because it guards everything deployed. The blind spot it removes is service-health-only monitoring — a model can serve two-hundreds at perfect latency while its accuracy quietly rots, and only model-level metrics see it happening.
Work through these
Service metrics vs. model metrics
Whether the service is responding is a different question from whether the model is still right. Systems that monitor only the first find out about the second from users.
Data drift, concept drift, and detection tests
The input distribution changing, and the relationship between input and outcome changing, are two different problems with two different responses. Tests exist for detecting each.
Delayed labels and proxy metrics
Often the true outcome arrives weeks after the prediction, so quality cannot be measured immediately. Choosing a stand-in measure that correlates with it is the practical answer.
Retraining triggers and automation
Deciding what triggers a retrain, and how much of it happens without a person. Retraining on a schedule regardless of need is a common and wasteful default.
Sign in to keep your progress.
Free resources
We haven't checked most of these for screen reader use yet.
Links last checked 29 Aug 2026.
Stuck here?
Ask a mentor. A real person answers, and they can see exactly which topic you're on. Usually within a couple of working days.
Checking your session…
Topics shown in module order.