The accountability gap

A piece of English silver carries several marks, and the useful thing about them is that they answer different questions and were introduced at different times, for different reasons.

The oldest of them is not the maker's. A statute of 1300, under Edward I, required that silver be tested against a standard and struck with a mark before it could be sold, and the striking was done by the wardens of the goldsmiths rather than by the person who made the article. The order of events matters more than the date. The independent test came first. The maker's own mark, the sponsor's mark, was made compulsory sixty-three years later, in 1363, and it was added for a reason that is worth stating plainly: so that an article which failed the test could be traced back to whoever had made it.

Seven centuries later the arrangement is unchanged in its essentials. An assay office is a body the maker does not control, does not pay for a favourable result, and cannot overrule. Its mark says a sample of this metal was tested by somebody other than the person selling it to you. The maker's mark sits alongside, and says who is answerable if the test was wrong. Two marks, two questions, two parties, and the whole apparatus exists because the medieval legislature concluded, correctly, that a maker's assurance about the purity of his own silver is not evidence.

Every assurance in enterprise AI today is a maker's mark.

The model card is written by the laboratory that trained the model. The benchmark score is produced by the laboratory, or on a benchmark the laboratory can see. The system card, the safety evaluation, the responsible-use policy and the transparency summary are all documents in which an interested party describes its own product. The vendor's SOC 2 report attests to the vendor's controls over the vendor's systems. The integrator's assurance covers the integrator's configuration. Inside the buying organisation, the validation pack is written by the team that built the thing, reviewed by a governance forum staffed by colleagues, and signed by somebody in the same reporting line.

Chapters four and five both arrived at a version of this observation from inside the Data layer, and named it the same way. A supplier attestation is a certificate rather than an audit. It tells you what the supplier says. The useful discipline is to read it, write down what it does not cover, and carry that list into the next conversation.

This chapter is about what happens when that list reaches the top of the organisation and there is nobody left to carry it to.

Above the Process layer, production output meets two audiences who did not build the system, cannot inspect it, and are answerable for it anyway. One is the board. The other is the regulator. Neither has an assay office. Both are being handed maker's marks and asked to treat them as evidence, and both, for the most part, currently do.

That is the accountability gap. It is not a risk, a concern or a maturity problem. It is a structural absence, in the specific sense that the institution which would close it has not been built yet.