The evidence trail nobody is building

Picture the inspection. It is 2027. Someone from the regulator asks: show me what this system did, on what data, which version of the model, and who signed it off. In a lot of companies, right now, the honest answer would be a long silence.

Picture the inspection. It is 2027. Someone from the regulator is sitting across the table, and they ask a question that sounds simple. This decision your system made, the one that went into the submission: show me what it did, on what data, which version of the model, and who signed it off.

In a lot of companies, right now, the honest answer would be a long silence.

The adoption has happened. Agents are running, copilots are drafting, models are quietly shaping decisions across the business. What has not happened, almost anywhere, is the boring part: the trail that lets you stand behind any of it after the fact. Most organisations can tell you what their AI is doing this morning. Very few can tell you what it did last March, and prove it.

Here is the strange part. The companies most exposed to this are the ones who already know better than anyone how to handle it. Pharma has run regulated production for decades on a simple, unforgiving principle: if it is not documented, it did not happen. Data integrity, audit trails, validation, ALCOA. Attributable, legible, contemporaneous, original, accurate. An entire discipline built around being able to prove, years later, exactly what happened and who was responsible. And then the same companies deployed AI as if none of those rules applied.

Proof, in an AI production system, is the evidence trail. It answers, for any output that mattered: what did the system actually do, on which inputs, using which model and which version, under whose authority, and can you run it again and get the same answer.

Attributable, so there is a named owner, not the AI did it. Traceable, so the inputs and the version are on the record, not reconstructed from memory. Reproducible, because a result you cannot reproduce is an assertion, not evidence. None of that is glamorous. All of it is load-bearing. It is the difference between a system you can defend and one you can only hope about.

The mistake almost everyone is making is to assume this can be added later. That when the regulator, or the board, or the lawyer asks, you will go and assemble the trail. You will not. You cannot reconstruct a chain of custody you never captured. The model has moved on a version, the data has been overwritten, the person who ran it has left. The evidence had to be created at the moment the work was done, or it does not exist.

This is the andon cord again. On Toyota's line, the record and the right to stop are built into the work, not bolted on after a defect has shipped. Proof works the same way. It is captured in production, as a property of the system, or it is not captured at all.

There is a version of the next few years where this stays invisible, because nobody has asked yet. That is the dangerous version, because it feels fine right up until the moment it does not. The first serious inquiry, the first product issue traced back to an AI decision, the first regulator who decides to make an example, and the question arrives. The companies that built the trail will answer it in an afternoon. The ones that did not will discover that we were moving fast is not a defence anyone accepts.

Proof is not paperwork, and it is not something you earn by being careful in hindsight. It is built in, at production time, while the work is happening, or it is gone. Every executive running AI today is making a quiet bet on which of those two it will be. Most of them do not yet know they have placed it.