Model drift
Definition
A deterioration or change in AI performance as the environment, data, users or conditions around the system change over time.
Put simply
The system may still be operating, but the world around it has changed.
What boards and leaders should understand
An AI system can become less reliable without anyone deliberately changing it.
Customer behaviour can change. Language can change. Source data can shift. New types of inputs can appear.
The underlying model itself may also change if a provider updates it.
This is one reason AI assurance needs to continue after deployment rather than ending with launch testing.
Questions to consider
- How would we know if an AI system remained technically operational but had become less reliable in the real world?
