A model is a depreciating asset
The day you deploy a model is the day it begins to decay. The world it learned from keeps moving, and the model does not move with it unless you make it. This is why the post-launch phase, not the build, is where most of the value of machine learning is won or lost. A model that was 94 percent accurate at launch and is 78 percent accurate a year later, with nobody watching, is not an asset. It is a liability that everyone still trusts, which is the most expensive kind.
Two kinds of drift, two different responses
It helps to separate data drift from concept drift, because the fixes differ. Data drift is when the inputs change: a new product line, a new customer segment, a sensor recalibrated, a currency added. The relationship the model learned may still hold, but it is now seeing inputs it was not trained on. Concept drift is deeper: the relationship itself has changed, so the same inputs should now produce different outputs. Fraud patterns shifting in response to your own defences is concept drift. Confusing the two leads to the wrong remedy, so the first job of a monitoring system is to tell them apart.
Monitor inputs and outputs, not just accuracy
Accuracy is a lagging indicator, and in many production settings you do not even have ground-truth labels until weeks later. So watch the leading signals. Track the distribution of incoming features and alert when they shift away from the training distribution. Track the distribution of the model's own predictions, because a model that suddenly predicts the positive class twice as often is telling you something even before the labels arrive. Where you do get delayed labels, close the loop and track true performance over time. The richest monitoring systems combine all three so a problem is visible long before it shows up in a quarterly metric.
Set thresholds that page a human, not a dashboard
A monitoring system nobody looks at is theatre. Drift detection has to be wired to alerting with thresholds tuned to the cost of being wrong. A low-stakes recommender can tolerate more drift before anyone is paged than a credit decision or a maintenance prediction on heavy machinery. The art is in setting thresholds that fire on genuine degradation without crying wolf, because an alert that goes off constantly is an alert that gets muted, and a muted alert is no alert at all.
Automate the response: retraining and rollback
Detection without a response is just anxiety. Mature MLOps automates the reaction. Continuous or scheduled retraining pipelines pull fresh, labelled data, retrain, and run the candidate through an evaluation gate before it ever touches live traffic. Equally important is the ability to roll back instantly when a new model underperforms, because retraining sometimes makes things worse and you need an undo button that works in seconds. In predictive-maintenance systems this is the whole design: survival models retrain as new telemetry arrives and write directly to the maintenance scheduler, so a flagged asset is slotted into the next workshop window rather than waiting for a human to notice.
Feature stores and reproducibility
Drift is far easier to manage when the features a model sees in production are computed exactly as they were in training. A feature store enforces that consistency and removes a whole class of silent bugs where training and serving quietly diverge. Add versioning for data, features, and models, and you gain the ability to reproduce any prediction the system ever made. That reproducibility is what turns an incident from a mystery into a fifteen-minute investigation.
Make decay visible and the rest follows
The organisations that keep AI healthy are not the ones with the cleverest models; they are the ones that made model decay impossible to ignore. Once drift is monitored, alerted, and answered with automated retraining and rollback, machine learning stops being a project that peaks at launch and starts behaving like infrastructure. That is the whole promise of MLOps, and it is what our MLOps and AI DevOps practice is built to deliver.