06-reference/research

high reliability acceptance gates agent contracts

2026-06-11·research-brief·source: deep-research
acceptance-gateshigh-reliabilityverificationcafcommissioning

What high-reliability acceptance gates teach AI-agent acceptance contracts

The question

How do high-reliability fields (aviation certification, nuclear, clinical trials, shipbuilding sea-trials) structure staged acceptance gates, and what transfers to AI-agent acceptance contracts? Extends the commissioning brief + acceptance-contract verify-skill hardening toward the gates end of the spectrum, with the CAF restructure (106 skills → ~15-20 objective+acceptance-test skills, HITL gates, manifest as state owner) as the live application.

What we already know (from the vault)

What the web says

Convergences and contradictions

Synthesis for RDCO

Staged gates are per-stage closure events plus one aggregate finding — not a thicker single DONE test. All four regimes decompose acceptance into many small, individually-closable gates (an ITAAC, a TIA, a phase readout, a builder's trial card) and then require a separate aggregate finding before authority transfers (52.103(g), type certificate, NDA approval, commissioning). For CAF this validates the manifest design and sharpens it: each phase's contract_validation entry is an ITAAC-style closure, and the build-manifest handoff at Phase 7 should be an explicit 52.103(g)-analog — a recorded finding that every gate cleared, made by the account lead, not implied by the last phase completing. The concrete upgrade is to adopt ITAAC's three-part format per contract item: design commitment (what the phase promised) / verification method (how it gets checked — re-run, inspect artifact, cross-reference) / acceptance criterion (the threshold). Today's completeness-score-only validation is one-third of that structure.

Hold-point placement follows error-propagation cost, not uniform spacing. Clinical phases escalate exposed population only after evidence at smaller scale; the FAA requires conformity inspection before official flight test because testing the wrong article voids everything downstream. The CAF HITL gates land correctly by this logic: Phase-1 framing sign-off (cheapest point to fix the most expensive class of error — the Sydney lesson) and Phase-3 classification sign-off (the routing decision every downstream lane inherits, structurally a certification-basis freeze). The rule to write down: place a HITL gate wherever the next phase multiplies the blast radius of an undetected error or makes it irreversible; everywhere else, an autonomous verifier suffices. The same logic gives a graduated-exposure ladder for new agent capabilities generally — pilot one collapsed phase skill (Phase B) before the full 106→15 collapse is the clinical-phase move, and running one client before /run-batch over a roster is the type-certificate-before-production-certificate move: "it worked once" and "it reproduces reliably" are different gates.

Who signs off: three roles, never collapsed into fewer than two parties. Every regime splits performer (licensee, builder, sponsor, applicant) from verifier (NRC inspector, INSURV, DSMB, FAA flight test) from authorizer (the Commission's finding, commissioning authority, FDA approval). INSURV's lesson is the sharpest: the verifier sits outside the producing chain entirely, reporting around the program office. For CAF: phase skills perform; Meta-Council + fresh-eyes verification check (the DO-NOT-COLLAPSE list is exactly right — collapsing m-1/m-2/m-3 into the phases they police would be putting INSURV on the shipbuilder's payroll); the human at HITL gates authorizes. And every regime makes closure an evidence artifact — the ICN, the TIA, the trial master file. The manifest's hitl_approvals records who/when; it should also record what evidence the approver saw (paths into contracts/ and assets/), so a resumed session or auditor can reconstruct the basis for each cleared gate. One cheap sequencing trick transfers directly: builder's trials before acceptance trials means the worker runs its own DONE self-check before the fresh-eyes verifier is spawned — same independence guarantee, fewer wasted verifier cycles.

For the acceptance-contract template itself, two amendments earn their keep. First, add a per-criterion verification-method field (the ITAAC third part) so the contract names how each DONE item gets checked, not just what it is. Second, borrow the arXiv paper's anti-Goodhart mechanism for any standing gate (verify-pdf-output's 12 checks, a future CAF regression suite): every HOLD/ROLLBACK-class failure feeds a new scenario into the test bank, so the gate hardens with use instead of being memorized. The failure-cost→gate-count ladder in the template is confirmed by every regime studied — these fields are all at the catastrophic end and run maximum gates; the transfer is not "add more gates everywhere" but "make each gate a named, recorded, independently-verified closure event."

Open follow-ups

Related

Sources