The model changed while you were congratulating yourself

One contradiction sits at the center of assessing anything intelligent: you judge its future behavior on the only evidence you have, how it behaved on the prompts you happened to try. With ordinary software that gap is bearable, because the code sits still. An AI system refuses to sit still, and that changes what a passing assessment is worth.

The system moves after you sign

You have your assessment at last. Every known failure mode named, a handful burning red, the sharpest testing aimed at them, a scope signed and a report pinned to the wall. For one clean afternoon it feels finished, and the instinct is to treat it as a certificate proving the model is safe.

Then the system moves, in ways a traditional application never would. The provider silently updates the model behind the same name, and last month’s validated answers come from different weights today. The world drifts past the training cutoff, so the model’s confident facts go stale. Your retrieval store ingests new documents that feed it outdated context. Nobody touched your code, and yet the thing your report described has ceased to exist.

History punishes those who trust a fixed picture of a moving thing, from both directions. Adam Osborne announced his brilliant next computers before they could ship; customers stopped buying the present for the promised future, and the company was bankrupt by 1983. Kodak made the opposite mistake with the same root. It so dominated film that it treated film as the future even after its own engineer built the first digital camera in 1975, and it clung to that triumph until the market moved on and it filed for bankruptcy in 2012. One trusted a future that had not arrived, the other a past already gone, both reading a picture that had stopped being true. A triumphant AI invites this error most of all: the better the model tests, the more certain everyone is that it is solved, and the less anyone dares re-examine it while, underneath the praise, it quietly drifts.

A report that never expires is a warning

So a serious method builds the expiry in. When the model version changes, the data shifts, or the deployment moves, the scope is marked invalid, and the report declares out loud that it must be redone before anyone leans on it again. Turn that into a health check and it inverts what reassurance looks like. A report still valid months later does not mean the AI is safe. It means either no one updated the model, no one noticed the provider did, or no one is watching the drift. For a system meant to keep being retrained, that green certificate reads correctly as a flat line on a monitor. The healthy signal is the opposite: an assessment that keeps invalidating itself, the trace of a living system watched by people still paying attention.

A report still valid months later does not mean the AI is safe.

The honest answer to the question you have circled since the first essay, the clean yes you wanted, is that an AI system is never finally safe and its assessment has no last page. There is always a new model version, a new document in the store, a new prompt no one has tried.

And somewhere in your catalogue of risks, in a failure everyone agreed was safely handled, the model just answered differently than it did yesterday.

Discover how Sogeti helps organizations continuously monitor, assess, and build trust in AI systems and secure your AI Trust & Assurance Assessment