The Council pattern is easy to sketch. Making it pharma-grade is the harder problem.

Following on from yesterday's post on Karpathy's LLM Council. Building the debate is straightforward. Building one your Quality and Compliance teams will sign off on is a different exercise. Here's the architecture I'd start from.

Have been thinking about this overnight. The Council pattern is easy to sketch. Making it pharma-grade is the harder problem.

Following on from yesterday's post on Karpathy's LLM Council. Building the debate is straightforward. Building one your Quality and Compliance teams will sign off on is a different exercise. Here's the architecture I'd start from.

Layer 1, the council members. Three to four frontier models with different training lineages, because the diversity is the point. A current shortlist for pharma work: Claude Opus 4.7, GPT-5, Gemini 3 Pro, plus a domain-specialised model where the use case warrants it. Avoid stacking models from the same provider, you lose the dissent signal.

Layer 2, the Chairman. Designate one model as synthesiser, ideally one with strong calibration and conservative hedging behaviour. Rotate Chairman by use case if you want to control for synthesis bias.

Layer 3, the orchestrator. Karpathy's repo running on OpenRouter works as a reference implementation. For production, expect to build on LangGraph or a custom layer with versioned prompts and explicit stage gating.

Layer 4, the anonymisation pass. Strip model identity before peer review. Without this, you get reputational halo effects and the whole exercise collapses into popularity voting.

Layer 5, the audit log. Every input, every individual response, every ranking, every Chairman rationale, captured and timestamped. This is what makes the pattern defensible under 21 CFR Part 11 and the EU AI Act, not a feature you can bolt on later.

Layer 6, the governance wrapper. PHI and PII redaction before any external API call. Data residency controls where the use case is GxP-adjacent. Model version pinning, because a silent provider upgrade in production breaks your validation.

Layer 7, the human checkpoint. Council outputs are inputs to a qualified reviewer, not final answers. The pattern augments expert judgement, it does not replace it.

The sequence matters. Most enterprise Council attempts I've seen treat governance as the last layer. In pharma it's the first.