The design space.

Fifth and final piece in the accountability series. The Named Owner argued that every AI deployment needs a person against the decision. You Cannot Acquire Accountability argued that the responsibility cannot be outsourced or merged away. The Judgement Muscle argued that the owner has to stay sharp enough to actually exercise it. The Cost of Holding It argued that doing all three properly is expensive, and that the organisations paying the bill cleanly are buying a position the rest of the market will eventually have to buy at a higher price. This piece is the operational answer to a question the first four were sometimes accused of avoiding. What does the discipline actually look like at 8:30 on a Tuesday with QA, Validation, IT, and a vendor on the call?

Four pieces in, the fairest version of the pushback I have received goes something like this. The accountability argument is fine in principle. Name the owner, hold the line, keep the judgement sharp, pay the bill. But what does any of this look like operationally? When the room contains the Head of Quality, the Validation lead, an IT director, and a vendor whose contract is renewing in six weeks, what is the actual conversation? Where does the pharma quality manual end and the new discipline begin?

I have been ducking the question because I did not have a clean answer. A piece dropped last week from a pharma quality voice that does. It is also, almost incidentally, the cleanest articulation I have seen of why the four pieces that preceded this one belong together rather than as separate arguments. The work is Nuno Valério's, in his Trust Architecture newsletter, building on a framework paper by Georgi Tancev submitted to IEEE Transactions on Engineering Management. The argument deserves wider circulation, and the rest of this piece is essentially my attempt to thread it into the series.

The wrong-shaped question

Valério opens with a meeting room in Darmstadt. A retrieval-augmented LLM deployment with human review at every step, audit trail, rollback procedure, the lot. A senior QA colleague asks the question every quality professional has been trained to ask of every system that touches a regulated record. Can you certify this is correct, or do I need to refuse to sign? He gives her the honest answer. No. She looks at him for a second and says: then what are we doing here?

That is the accountability series in one exchange. The Named Owner is the seat she occupies. You Cannot Acquire Accountability is the reason she cannot pass the question to a committee or a vendor. The Judgement Muscle is the discipline she is being asked to exercise in real time. The Cost of Holding It is what the organisation is paying her to sit there and ask. And the question she asks, can you certify this is correct, is the question the whole regulated industry has been trained to ask. It is the right question for deterministic software. She is applying it, correctly by training, to a probabilistic system. The conversation cannot move forward until the shape of the question changes.

Three classes of failure

Tancev's framework, as Valério summarises it, partitions LLM failures into three classes. The first is engineering problems. Prompt sensitivity, context degradation, vendor drift. These are reducible with effort. The second is sociotechnical. Automation bias, evaluation gaps. Reducible with organisational discipline. The third is the one that matters here. Hallucination, semantic quantifier failure (the model that confidently states all batches comply when only some do), and the mathematical fact that softmax decoding assigns non-zero probability to every possible output at every step. No instruction can categorically exclude anything. Tancev calls this class provably irreducible.

Valério is honest enough to flag that the word provably is doing more work than the underlying theorems fully earn yet. Two of the three formal results he cites are still in peer review. The softer version of the claim is the one I would stand on operationally. The engineering ceiling on these failures is high enough, and the testable input space small enough relative to the deployable input space, that we should treat them as if they were irreducible for the purposes of governance. Same operational conclusion. Less dependence on theorems that have not yet settled.

The point is the conclusion. If a class of failures cannot be engineered down meaningfully from where it sits, then governance built on the assumption that it can be is governance built on a fiction. Most pharma AI governance today, the GAMP AI guide, the seven-control-layer frameworks, the draft Annex 22, is built on exactly that assumption. They are good frameworks for the reducible failures. They are quiet on the part that will not reduce.

The map has been in the quality manual the whole time

Here is where the argument turns. Pharma already knows how to govern this kind of system. It has known since 2009. ICH Q8 talks about a design space. The industry does not certify that a manufacturing process will never produce an out-of-spec batch. It defines the multidimensional space of input parameters within which the process operates reliably. It characterises what happens at the edges. It monitors for drift. It accepts that operating inside the space is a bounded probabilistic claim with documented controls at the boundary. Fifteen years of small-molecule discipline have made this second nature for anyone who works in quality.

Valério's claim, which I think is exactly right, is that the right discipline for LLM governance is the same discipline. Not prove it will never hallucinate. Define the scope where hallucination consequence sits inside tolerance, monitor for the cases that fall outside, design the verification layer to catch what slips through. This is not a downgrade in rigour. It is the right shape of rigour for the system being governed, finally arriving.

Three moves at 8:30 on a Tuesday

With that frame in place, the operational conversation gets simpler. Three moves, all of which Tancev formalises, and all of which pharma quality professionals will recognise as the same logic applied to a different object.

First, classify by consequence rather than by model. The same LLM is a low-risk drafting aid when it summarises meeting notes and a high-risk system when it drafts the clinical summary for a marketing authorisation. The named owner is not governing the model. They are governing the use case. Tancev's version of this is the LLM Deployment Integrity Level, four tiers running from low-consequence advisory through to containment-engineered autonomous action. The classification scheme is new. The move of mapping consequence to control intensity is what GAMP already does for everything else.

Second, document the failure boundary explicitly. Which failure modes are present in this deployment. Which are reducible by engineering. Which are not. What scope restriction or independent verification addresses the ones that are not. This goes in the validation dossier as a boundary specification, not as a performance metric. It is the closest thing pharma has to honest validation language for a probabilistic system, and it is what the named owner signs against. The boundary specification is what makes the cost of holding accountability visible on paper rather than implicit in the line spend.

Third, the irreducible failures get scope restriction, not better testing. If hallucination cannot be driven to zero, the LLM does not author universal claims in regulatory submissions. It does not summarise a study without independent verification of every numerical value. It does not issue anything that looks like a release authorisation. The scope is what matters. Not the model card. Not the benchmark. The discipline is to decide in advance where the model is not permitted to operate, and then to defend that line when commercial pressure makes the case for relaxing it.

Two data points worth considering

Two recent items worth carrying out of Valério's piece, because they show the gap between framework and practice closing from both sides. Microsoft Research published a benchmark called DELEGATE-52 earlier this year. Nineteen frontier models run through 52 multi-step professional document editing workflows. Average corruption rate by end of workflow, across all models, 25%. The errors were sparse, severe, and silent. Self-verification by the same model class did not catch them. For any agentic pipeline assembling regulatory documents through multiple LLM-assisted editing passes, and there are several of these in flight inside the top twenty pharmas right now, the implication is direct.

Separately, a study found that 68% of production agent teams have already capped autonomy at ten steps before mandatory human intervention. Not because a governance framework told them to, but because the failure rate at longer horizons made the systems unusable. Practitioners arrived at the answer empirically, without theoretical grounding. The framework, in other words, is catching up to what the practitioners already learned the hard way. That is usually a good sign that the framework is converging on something true.

What this series attempted to decode

Five pieces. One argument, told in five parts.

The Named Owner established that AI accountability has to sit with a person rather than a function. You Cannot Acquire Accountability established that the responsibility does not transfer with a restructuring, an outsourcing arrangement, or an acquisition. The Judgement Muscle established that the owner must remain capable of exercising the ownership, and that strong model performance is the anaesthetic that quietly erodes the capacity to do so. The Cost of Holding It established that holding accountability properly costs real money in places the accounting structure does not name, and that the organisations paying the bill cleanly are buying a position the rest of the market will eventually have to buy at a higher price. This piece establishes the last part. The discipline of doing all four properly is not new. It is the design-space discipline pharma quality has been doing on small molecules since 2009, applied to a different object.

The five arguments are sequential, but they collapse into a single instruction. Name the owner. Hold the accountability. Keep the owner awake. Pay the bill. Then run the deployment inside the design space the owner can defend, with the boundary written down and the irreducible failures kept outside it.

That is the whole job. The map for AI governance was not missing. It has been sitting in the quality manual the entire time. The remaining work is to recognise it, transpose it, and stop asking probabilistic systems to answer deterministic questions.

This concludes the five-part accountability series. The Named Owner, You Cannot Acquire Accountability, The Judgement Muscle, The Cost of Holding It, and The Design Space are on the Wild Geese site.

With thanks to Nuno Valério, whose Trust Architecture newsletter supplied the ICH Q8 transposition this piece is built on, and to Georgi Tancev, whose framework paper to IEEE Transactions on Engineering Management gave the argument the formal scaffolding it had been missing. The Darmstadt anecdote, the three-class failure partition, and the LDIL classification are theirs. The series-closing argument and the pharma-commercial extension are mine.