Audit after the fact is too late

The call comes eighteen months after the work was done. Someone needs to know exactly what the system did, and why, and who stood behind it. So the team sits down to reconstruct it. The model is three versions on. The data has been overwritten twice. The person who ran it has left. The retrieval corpus is unrecognisable. They are not auditing a decision any more. They are excavating a ruin.

The mistake underneath this is treating the evidence trail as something you can assemble on demand. You cannot. A trail is not a thing you find later. It is a thing you either captured at the time or did not, and by the time anyone asks, the conditions that produced the output are already gone. They do not come back because you need them to.

Manufacturing settled this argument a long time ago. A batch record is written as the batch runs, each step recorded as it happens, by the person doing it, while it is still true. Nobody in pharma would dream of running a batch and then writing the record next year from memory. The whole discipline exists because everyone learned that memory is not evidence. And yet that is precisely how most AI is being operated right now. Do the work today, and hope you can explain it later.

Capturing at production time means the evidence is a property of the system, not a chore for a person. Every consequential run logs itself, as it happens, with its inputs, its model version, its outputs, and its human checkpoints. The record is written while the work is being done, and then it is closed and immutable. Not editable afterwards, not summarised down once the detail gets inconvenient, not dependent on anyone remembering to go back and document it.

This is the andon cord again. On the line, the right to stop and the record of what happened are built into the work itself, because bolting them on after a defect has already shipped is worth nothing. Proof works the same way. It is captured live, or it is not captured.

There is no such thing as a retrospective evidence trail. There is the trail you built while the work was happening, and there is the silence where it should have been. The audit does not create the evidence. It only finds out whether you did.