There is no verification ratio — the published analogues span four thousand-fold, and the analogue closest to agent work products buys its throughput by sampling judgment rather than checking the artifact
The question
"What is the real verification ratio - how many agent-produced work products can one senior professional sign off per week at professional-liability standard? Use analogues where published: audit review rates, radiology second-read throughput, code-review throughput."
Filed as a derivative of [[2026-07-27-psf-pyramid-economics-post-agent-analyst-tier]], whose central finding was that the scarce layer in professional services moved from execution to verification. That brief's "new leverage ceiling" argument rests entirely on this number, and it closed by noting no professional-services firm has published it.
⚠️ Verification status of the numbers in this brief
Source-verification pass completed 2026-10-08. The brief was originally closed out before its primary-source pass: the three analogue researchers (audit, radiology, code review) had been dispatched but had not returned. All three have since returned with directly-retrieved sources, and a targeted verification pass re-pulled the brief's most load-bearing figures from their own documents. Every number below still carries one of three labels, and the labels remain load-bearing:
- [SESSION-VERIFIED] — the source document was fetched and read on 2026-10-08, either by this verification pass or by one of the three analogue researchers working the same session. Quotable.
- [SEARCH-RELAYED] — surfaced by web search as a summary of a named source, but the source page was not fetched and read. Treat as a lead.
- [RECALLED] — drawn from knowledge of the published literature, with source and year named so it is checkable, but not retrieved. Do not quote these externally until someone opens the source.
Now verified. All three analogues have a primary-verified spine. Code review: the METR RCT and its 2026 sequel, Cohen's Cisco study (article and book chapter), the Google ICSE-SEIP paper. Radiology: NHSBSP Publication No 59, Partridge 2024 reading times, the Cornford 2021 volume-to-quality curve, MASAI, PRAIM, and the Dratsch automation-bias trial. Audit: the text of AS 1220 itself, PCAOB's own post-implementation white paper with measured EQ-reviewer hours, the Korean EQCR-hour disclosure series, and FRC partner-load data.
Still unverified, and still not quotable. Fagan 1976 (paywalled; the circulating LOC/hour rates are secondary restatements that disagree with each other); Goodwin & Wu and López & Peters on audit-partner busyness (paywalled); McDonald et al. on cross-sectional imaging workload; Rigby & Bird FSE 2013 (paywalled — available only as quoted inside the Google paper, which is peer-reviewed secondhand rather than primary); NBER w35275; and all four vendor-telemetry figures. There is also no clean published mean of engagements per signing audit partner anywhere — the figure below is derived, not reported.
Five figures were corrected rather than confirmed, and one headline was downgraded. Wrong as stated: the Cisco LOC/hour threshold; the NHSBSP 5,000-case framing; MASAI's workload reduction; PRAIM's reading-time comparison; and the audit analogue's engagements-per-partner-week rate, which had no retrievable source at all. Downgraded: METR's 19% slowdown, which is correctly stated but which METR itself has since withdrawn confidence in. Each is marked CORRECTED inline with its prior value. That hit rate — five wrong in roughly twenty checkable figures — matches the 2-in-10 finding-level defect rate in [[2026-10-08-finding-level-citation-audit-arxiv-gloss]] almost exactly.
No number here should be put in front of a client, a prospect, or a public post unless it is SESSION-VERIFIED. This is exactly the false-verified-stamp failure mode in feedback_workflow_agent_output_integrity, so the labels stay in the document rather than being smoothed out.
What we already know (from the vault)
- The vault's own answer is 2-3, and it is about accountability relationships, not work products. [[2026-06-03-parallel-agent-fde-capacity]] found the practitioner ceiling for actively supervised agents is 2-4 (Hashimoto's "mayor of two"; Addy Osmani's "3-5 is the sweet spot"), and concluded RDCO's realistic concurrent-client N is 2-3 with one in active build. Critically, it drew a distinction this question tends to erase: background fan-out and foreground judgment work do not scale the same way. Machine-validated background fan-out is bounded by token cost; foreground work that carries professional liability is bounded by one human's attention.
- The mechanism that degrades the reviewer is named and is about trust, not time. Same brief: "your attention depletes, your trust in the output inflates, and every context switch makes both worse" — sustained review "quietly shifts from deep evaluation to surface scanning." The ratio is therefore not a fixed capacity; it is a capacity that silently looks intact while quality falls.
- The best hard number in the vault is an attenuation ratio, not a throughput ratio. [[2026-06-07-alphasignal-async-agents-release-bottleneck]] logs NBER working paper w35275 ("Writing Code vs. Shipping Code," MIT/Wharton, ~100k GitHub developers + Microsoft telemetry): synchronous AI coding tools produced a 741% increase in raw lines of code and +65% pull requests, but only ~20% more actual releases; cumulative-commit gains of 180% attenuate to ~50% at project level and ~30% at releases, with an elasticity of substitution of ~0.25. [RECALLED via the vault note, which itself relayed it from AlphaSignal — the NBER paper has never been pulled directly into the vault.] This is the single most useful quantity we hold: it measures the size of the verification choke, which is a different and more honest thing than a ratio.
- The vault also holds the most aggressive counter-claim, and it is self-reported. [[2026-01-23-every-ai-code-reviews]] — Kieran Klaassen (Every/Cora) reviewed a bug touching 27 files and 1,000+ lines in 15 minutes of human decision-making, by routing it through 13 specialized AI reviewer agents first. Implied rate: roughly 4,000 LOC per human-hour. Hold that against the code-review analogue below; the two cannot both be right about the same activity.
- RDCO already has a pricing position that does not depend on this number. [[2026-06-13-managed-ai-data-services-pricing-models]] and [[2026-06-07-solo-fde-contract-structures]] both put a fixed backbone first with outcome pricing as capped upside. That stance survives whatever this brief finds, which is worth saying plainly: the verification ratio is a capacity question, not a pricing question.
What the web says
Analogue 1 — audit partner sign-off. The unit is an engagement, and the find of this pass is that liability-grade sign-off is cheap as a share of the work product, not expensive.
- What AS 1220 actually requires. PCAOB AS 1220, Engagement Quality Review (adopted as AS No. 7 by PCAOB Release No. 2009-004; amendments to .04, .05, .10 and .15 effective 2026-12-15) mandates an EQR and concurring approval of issuance on every issuer audit, interim review, and specified broker-dealer attestation engagement (.01). The requirement is an evaluation of the engagement team's significant judgments and the conclusions reached (.02, .09, .10) — explicitly not a re-performance of the audit. The reviewer "should not make decisions on behalf of the engagement team or assume any of the responsibilities of the engagement team," and the engagement partner remains responsible (.07). The reviewer must be a partner or equivalent and may come from outside the firm (.03), and cannot have been engagement partner on either of the two preceding audits (.08). Concurring approval may be given only if, after due professional care, the reviewer "is not aware of a significant engagement deficiency" (.12) — and the firm may not let the client use the report until that approval exists (.13). [SESSION-VERIFIED — standard text read 2026-10-08.] AS 1220 is a requirements standard and sets no rate; it should never be cited for a throughput number.
- The measured rate, and it is the most important new number in this brief: 66 EQ-reviewer hours per engagement. PCAOB's own post-implementation review of AS 1220 reports, for US global-network-firm issuer audits FY2008-2013, that "the average reviewer hours per engagement is 66 (68) hours, comprising approximately 1 percent of total audit hours," rising with issuer size (market cap <$700M → 53 hrs; $700M-$5B → 67 hrs; >$5B → 103 hrs). Average total audit hours per engagement in the same sample: 11,277 (824 for triennially-inspected non-affiliated firms). [SESSION-VERIFIED.] One statistic caution: 66 ÷ 11,277 is 0.59%, so PCAOB's "approximately 1 percent" is a mean of ratios while 11,277 is a mean of hours — do not present the two as the same statistic. No PCAOB ex-ante rulemaking hours estimate could be found; this 2018 white paper is the only verifiable PCAOB hours figure, which supersedes this brief's original guess that the number lived in a rulemaking economic analysis.
- Korea, the one jurisdiction mandating EQR-hour disclosure, puts it slightly higher and confirms the order of magnitude: EQCR hours are 1.8% of total audit hours (mean; 1.5% median; range 0.3-6.6%; N=4,580 firm-years FY2014-2016), against Korean total audit hours of 715 mean / 651 median. [SESSION-VERIFIED.]
- Review hours are strongly sublinear in artifact size. Total audit hours span 824 → 11,277 across firm tiers, a ~14x range, while EQ-reviewer hours span 53 → 103, a ~2x range. That is the opposite of the intuition that a bigger artifact needs a proportionally bigger review — and it is only possible because four tiers of filtering (team sampling → staff review → manager review → partner review) sit between the underlying transactions and the EQ reviewer's 66 hours.
- No clean published engagements-per-signing-partner figure exists, and this brief should not state one. The best available is derived from FRC primary data: Big Four UK 2024 had 883 public-interest-entity audit clients across 1,305 Responsible Individuals = 0.68 PIE audits per RI (1.52 per audit principal), with average audit fee income of £2.31m per RI (£2.82m Big Four). PIE clients only, excluding non-PIE statutory audits, so this is a floor, not a count. Separately, 89% of Japanese listed-firm audits have a lead partner carrying at least one other client (N=57,513 firm-years). [SESSION-VERIFIED — but derived, not reported.] This replaces the brief's original "low single digits to roughly ten per year" and the "~0.1-0.2 signed opinions per partner-week" converted from it, neither of which could be confirmed. Goodwin & Wu and López & Peters stay [RECALLED] — paywalled, not retrieved. Also not verified and not to be used: a Finnish "3 to 6 public clients" busyness figure and a 2,283-partner Form AP sample.
Analogue 2 — radiology second reading. The unit is a study, the rate is per-hour, and this is now the best-measured analogue on the board.
- CORRECTED — the NHSBSP 5,000-case figure is a competence floor, and the programme sets no rate cap at all. NHSBSP Publication No 59 §3.4(d) requires readers to "undertake a minimum of 5000 screening and/or symptomatic cases per year." Double reading (§3.5.2) is "recommended and should be considered mandatory in units that have moved entirely to digital mammography" — recommended, not universally mandated, which this brief previously reported as a blanket mandate. A grep of the full guidance found no cases-per-hour and no per-session cap anywhere: the programme regulates annual volume and outcome metrics, never rate of work. [SESSION-VERIFIED.] The brief previously glossed 5,000/year as "roughly 100 cases per reader-week" and treated it as a throughput figure. At the measured reading time below, 5,000 cases is about 82 hours of reading per year — under two hours a week, so it bounds nothing. Actual English per-reader volumes span 1,000 to 45,000 cases per year across 759 readers. [SESSION-VERIFIED.] Any use of 5,000/year as a capacity ceiling inverts its meaning by roughly an order of magnitude.
- Measured screening read time: 0.98 ± 0.30 min per case for 2D FFDM (~59 seconds, ~61 cases/hour) and 2.09 ± 0.64 min for DBT (~29 cases/hour) — exactly double, p<0.001 (48 readers, 34,210 FFDM cases, PROSPECTS trial inside the NHSBSP). Radiologists and radiographer readers did not differ (2.06 vs 2.14 min, p=0.71). Experience buys about 22% and plateaus inside nine months. [SESSION-VERIFIED.] This confirms the brief's previously recalled claim that tomosynthesis roughly doubles per-case reading time, and supplies the hard per-case number the brief lacked.
- Caveat to carry rather than bury: 59 seconds is batch pure-reading clock only. It excludes arbitration, consensus conference and dictation — and NHSBSP routes disagreements to a third arbitrating reader, which is where the expensive judgment actually lives. Any per-case cost of a second opinion built on 59 seconds omits the entire disagreement-resolution machinery, which is the part of the process most analogous to what a senior reviewer of agent output actually does.
- The volume-to-quality curve, and it is the most useful single finding in this analogue. Readers at 2,000 / 5,000 / 10,000 cases per year had PPV of 12.9% → 14.4% → 17.0%, driven entirely by recall rate falling 5.8 → 5.3 → 4.5 per 100 — while cancer detection rate stayed flat at 7.6 / 7.6 / 7.7 per 1,000. [SESSION-VERIFIED.] Five times the reps bought specificity, not sensitivity: higher throughput made readers less twitchy, and found no additional real signal. The thing that improves with volume is the false-positive rate, not the catch rate.
- CORRECTED — both headline AI-workload numbers are much weaker than their circulation implies. MASAI's "44.3% workload reduction" is a count of screen readings (46,345 vs 83,231), not clock time; the trial measured no reading times at all. Its cancer-detection difference (6.1 vs 5.1 per 1,000) was p=0.052 — not significant. PRAIM's 39s-vs-67s figure is a within-arm easy-versus-hard comparison, not AI-assisted versus unaided: the paper states verbatim that reading time "was measured in the AI group only as it was technically not possible to measure this in the control group," and its headline 56.7% workload reduction is an explicitly "fictitious AI automation scenario" simulation equal to the triage fraction (262,055 of 461,818 exams tagged normal), not a measured saving. [SESSION-VERIFIED.] This brief previously carried MASAI at "~44% workload reduction and ~29% higher detection" as if both were measured and significant. Its instinct to warn against treating MASAI as transferable was correct, and now rests on firmer ground: the time saving was never measured in the first place.
- Where AI sits in the SECOND-READER slot, per-case time goes UP. Lung-nodule CT: +33 to +41 seconds per case with AI as second reader, against a 77-104 second saving when the same AI runs as a first-pass/pre-screening reader. CT-colonography CAD-as-first-reader was a wash (243 → 239 s) and faster only when CAD produced ≤20 marks. For the most expert reviewer the measured benefit is approximately zero (chest X-ray: pulmonologists 67 → 30 s, radiologists unchanged at 61 s). [SEARCH-RELAYED.] Two principles fall out, and both transfer: reviewing machine output is faster only when the machine is terse — verbose machine output costs more to review than no machine at all — and the more expert the reviewer, the smaller the benefit.
- Automation bias is the strongest reliability number available, and experience does not protect against it. 27 radiologists were given staged incorrect AI BI-RADS suggestions. Accuracy when the AI was correct versus incorrect: inexperienced 79.7% → 19.8%; moderately experienced 81.3% → 24.8%; very experienced 82.3% → 45.5% (all p≤.003, r=0.93-0.97). [SESSION-VERIFIED.] The most experienced readers still lost nearly half their accuracy when the machine was confidently wrong. This is the measured form of the trust-inflation mechanism that [[2026-06-03-parallel-agent-fde-capacity]] could only name qualitatively, and it is the single most transferable reliability finding in this brief.
- A closer analogue than mammography, for a complex multi-finding review. Lung-screening LDCT reads at 9.1 ± 2.3 min per case independently, falling to 7.2 ± 1.0 min with a second reader — but sensitivity rising 77.5% → 90.8% came with false positives per case rising 0.33 → 0.60. [SESSION-VERIFIED.] A second read is a purchase of recall paid for in noise, and this is the per-case figure to use if one number is needed for a heterogeneous multi-finding artifact.
- No radiologist-performance-after-AI-withdrawal study exists. Verified negative. The only measured deskilling signal anywhere is from endoscopy: unaided adenoma detection fell 28.4% → 22.4% after CADe adoption — but it is observational, ~19 endoscopists, 4 Polish centres, and adenomas-per-colonoscopy did not change. [SESSION-VERIFIED — a striking signal, not causation.] Nothing here should be used to imply that radiology deskilling evidence exists.
- McDonald et al. (Mayo Clinic, Academic Radiology, ~2015) on cross-sectional imaging workload: volume growth implies a radiologist must interpret one image every three to four seconds across an eight-hour shift. [RECALLED — not retrieved this pass.]
Analogue 3 — code review. The unit is a change, and the best-grounded weekly throughput number in this entire brief lives here.
- Google: the median developer reviews 4 changes per week. Verbatim: "the median for changes reviewed by developers per week is 4, and 80 percent of reviewers review fewer than 10 changes a week," against a median authoring rate of about 3 changes a week. Median lines modified per change is 24; over 10% of changes modify a single line; over 35% touch only one file; median reviewer count is 1, and fewer than 25% of changes have more than one reviewer. Dataset: approximately 9 million reviewed changes created by more than 25,000 authors and reviewers, January 2014 to July 2016, plus 12 interviews and 44 survey responses. [SESSION-VERIFIED — paper PDF read 2026-10-08.] This is the number to use as the brief's headline code-review figure. Two cautions: the paper's "median latency for the entire review process is under 4 hours" is wall-clock latency, not reviewer-minutes, and is routinely misread as effort; and Google's throughput is bought by making the unit tiny, not by making review faster.
- CORRECTED — the Cisco/SmartBear LOC-per-hour threshold. The study is real and larger than the brief implied: Cisco Systems MeetingPlace group, 10 months, 2,500 reviews, 3.2 million lines of code, 50 developers on three continents, published 2006 by Jason Cohen (who founded SmartBear, which sells the review tool used — a disclosed conflict worth keeping). This brief previously said defect density "collapses above roughly 300-500 LOC per hour." What the study measured is a specific pair of thresholds: reviewers slower than 400 lines/hour were above average at uncovering defects, and above 450 lines/hour defect density was below average in 87% of cases. The commonly restated "500 LOC/hour" traces to SmartBear's undated vendor page citing unspecified "SmartBear research," not to the Cisco data. [SESSION-VERIFIED — Cohen's published article and the book chapter both read 2026-10-08.]
- The 200-400 LOC cap and the 60-90 minutes are prescriptions, not measurements, and the brief now labels them as such. Cohen's own write-up gives them as recommendations stated separately from the data: "review fewer than 200-400 LOC at a time," "aim for an inspection rate of less than 300-500 LOC/hour," and "spend less than 60 minutes reviewing" — with 60-90 minutes cited as the general interval after which concentrated effort degrades, not a measured review threshold. Defect density did decrease sharply above 200 LOC under inspection. [SESSION-VERIFIED.] Do not use SmartBear's widely-laundered "a review of 200-400 LOC over 60 to 90 minutes should yield 70-90% defect discovery" — no source is cited for it anywhere, and it is the figure most often passed off as a research finding.
- Cisco's strongest measured result is the one nobody quotes, and it is the most transferable thing in this brief. Defect discovery rate is roughly constant — about 13 per hour, with 94% of reviews under 20/hour (Cohen's article summary renders it as ~15/hour) — regardless of how much code is in front of the reviewer. [SESSION-VERIFIED.] Enlarging the artifact therefore does not increase the problems found; it increases the problems missed. Sign-off throughput scales with reviewer-hours, not with artifact volume, and batching more agent output into one review buys no additional coverage.
- And the study most cited for a per-reviewer rate is the study that measured that there is no stable per-reviewer rate. Best fit for the sampled reviewer's inspection rate was R²=0.29, ANOVA assumptions were unmet, and the chapter concludes rates "vary widely, even with similar authors, reviewers, files, and size of the review." [SESSION-VERIFIED.] The most-cited number in this literature comes from the study that found the number is not a constant, which is direct support for this brief's title claim.
- Author preparation is the one tooling-independent intervention in the literature, and its agent analogue is exact. Cisco reviews carrying at least one author-annotation comment never exceeded 30 defects/kLOC, versus up to ~130 for unannotated changes. The chapter explicitly declines to separate causation from self-selection of meticulous developers. [SESSION-VERIFIED — correlational, not causal.] An agent that annotates its own work and flags its own uncertainty is the same move, and it is the cheapest lever on the board.
- Fagan inspection (IBM, 1976), the ~125-150 LOC/hour conservative bound: [RECALLED — and now doubly suspect.] The original is paywalled and could not be tied to any specific rate in its own text; the circulating figures disagree with each other (125 vs 130-150), and one source attributes 100-150 to Radice rather than Fagan. Treat as folklore until someone opens Fagan 1976.
- CORRECTED and sharpened — reviewing AI output appears slower per unit, but METR has disowned its own headline. The METR randomized controlled trial (AI tools available Feb-Jun 2025; 16 experienced open-source developers, 246 tasks, repositories >22k stars and >1M LOC, developers with ~5 years on those same projects, paid $150/hr, each issue randomly assigned to allow or disallow AI) measured tasks taking 19% longer with AI — a slowdown — with a confidence interval of +2% to +39%. The perception gap is wider than this brief originally stated: the same developers forecast a 24% speedup beforehand and still estimated a 20% speedup after finishing, a roughly 39-point gap between felt and measured. Economics experts predicted 39% speedup, ML experts 38%. [SESSION-VERIFIED.] This must not be cited as settled. METR's own follow-up (February 2026; 57 developers, 143 repositories, 800+ tasks) calls the new data "an unreliable signal of the current productivity effect of AI tools" and "only very weak evidence," because a significant number of developers declined to participate rather than work without AI (30-50% withheld particular tasks) and pay was cut from $150 to $50/hr; the returning-developer subset came out at -18%, i.e. a speedup, with a confidence interval of -38% to +9%, and METR is redesigning the experiment. [SESSION-VERIFIED.] Sign convention: positive = took longer. Several secondary write-ups flip the signs and report METR as having reversed itself; it did not — it withdrew confidence, not direction.
- No study holds artifact size constant and compares reviewer-minutes on human-written versus AI-written code. That is the honest state of the literature, and it is the crux of this brief's central question. Indirect signals only: ~45% of merged Claude Code-authored PRs required further human modification (567 PRs, 157 projects, arXiv 2509.14745); ~10% of agent-generated methods were deleted during review; GitClear reports duplicated code blocks up 8x in 2024. [SEARCH-RELAYED — the GitClear figure is vendor-published on their own customers' repositories. Do not use the "churn 3.3% → 7.1%" figure; its year labels do not reconcile with GitClear's own dataset.]
- The unit caveat that makes every figure above non-transferable, and it is the most important line in this section. Every code-review number here measures a human reviewing a diff that has already passed a compiler, a linter, a test suite and — at Google — presubmit CI and hundreds of static analyzers. The reviewer handles only a residual class of problems. An analytical artifact arrives with none of that: the human is compiler, test suite and reviewer at once. Transferring "4 reviews per week" to unverified analytical work products over-estimates sustainable throughput by roughly the size of the CI layer you do not have.
- Rigby & Bird (FSE 2013), the source of the comparison figures for AMD (44 lines), Lucent (263 lines) and Microsoft, is paywalled; its numbers are available only as quoted inside the Google paper. [Peer-reviewed secondhand — better than blog-grade, not primary.]
- Vendor telemetry points the same direction. LinearB's analysis of ~8.1M pull requests reports AI-generated changes waited ~4.6x longer for review to begin and were accepted ~32.7% of the time versus ~84.4% for human-written changes; Faros AI reports high-AI-adoption periods merged ~98% more PRs while review time rose ~91% (one summary puts median review time up 441.5% against 33.7% task-throughput growth); a Digital Applied survey (n=2,847 developers, Q1 2026) reports reviewing AI-generated code consuming a median 11.4 hours per week, rising to 14-16 for heavy agentic users. [SEARCH-RELAYED — all four are vendor-published with a commercial interest in review tooling; none were fetched. Treat as directional only.]
- Nobody has published the number the question asks for. An extended search for a per-reviewer weekly capacity figure for agent output returned only per-hour rates, single-case anecdotes, and vendor ratios. [SESSION-VERIFIED — this is a verified negative result, and it is the most reliable finding in this brief.] A "61% of AI-agent PRs received no human review" figure circulates attributed to a 2026 peer-reviewed study and could not be traced past secondary reporting. [UNVERIFIED — do not use.]
Convergences and contradictions
- Convergence on direction, total divergence on magnitude — and the spread is now measured, not estimated. All three analogues agree that verification capacity is finite, degrades with throughput, and degrades invisibly. But stated as reviewer-time per unit — which is the honest common denominator, and what the brief originally lacked — they are nowhere near each other: 0.98 minutes for a screening mammogram, 9.1 minutes for a lung-screening LDCT read, and 66 hours for an audit engagement's quality review. That is roughly four thousand-fold, and the spread is not noise — it is the whole answer. The brief previously stated this as "~0.1-0.2 engagements per partner-week, ~100+ studies per reader-week, ~6-11 pull requests per reviewer-week"; the first and third were unverified and the second inverted a regulatory floor, so all three have been replaced. The one verified units-per-week figure in the literature is Google's: a median developer reviews 4 changes per week, 80% fewer than 10.
- The ratio is a property of the unit, not of the reviewer — and this pass adds a fourth mechanism to the three the brief named. What predicts a high ratio is (a) unit homogeneity, (b) a trained perceptual or statistical prior over the unit's error distribution, (c) a machine pre-filter that catches mechanical defects before a human looks, and — new, and the one the audit data forced — (d) sampling the judgment layer instead of verifying the artifact. Mammography has (a), (b) and increasingly (c). Code review has (c) and partly (a). Audit has only (d), and (d) alone is enough to keep liability-grade sign-off at 0.6-1.8% of engagement hours.
- Review effort is sublinear in artifact size, in two independent analogues. Audit total hours span ~14x (824 → 11,277) while EQ-reviewer hours span ~2x (53 → 103). Cisco found defect discovery rate roughly constant at ~13/hour regardless of how much code was in front of the reviewer. Both say the same thing from opposite ends: a bigger artifact does not get a proportionally bigger review — it gets the same review and more escapes.
- Contradiction, and it is the one that matters: the question's framing assumes agent-output verification is throughput-comparable to human-output verification, and the evidence runs the other way — though more weakly than this brief originally claimed. Human work products arrive with legible authorial uncertainty — hedges, flagged assumptions, visible shortcuts, an identifiable skill level that tells a reviewer where to look. Agent output arrives uniformly confident and uniformly fluent, which removes the reviewer's cheapest triage signal: there is no "this paragraph reads like someone was guessing." The measured consequences point one way, and the radiology data now supplies the cleanest version of it: where a machine occupies the second-reader slot, per-case review time goes up (+33 to +41 s); where it occupies the first-pass slot, time goes down (77-104 s saved) — and CAD was faster only when it emitted ≤20 marks. Verbose machine output costs more to review than no machine at all. METR's +19% (CI +2% to +39%) points the same way, and LinearB's 4.6x review-start delay and 32.7% acceptance rate and Faros's 91% review-time increase are directionally consistent but vendor-published. Per-unit verification of agent output is plausibly slower, not faster, than verification of human output at the same stated quality bar. Two honesty notes that weaken this: no study holds artifact size constant and compares reviewer-minutes on human- versus AI-written work, and METR has withdrawn confidence in its own headline (its returning-developer subset came out at -18%, a speedup, CI -38% to +9%). The direction survives; the magnitude is not established. The parent brief's leverage-ceiling argument needs the opposite direction to be true, and it is not getting it.
- Contradiction inside the vault: Every's ~4,000 LOC per human-hour against Cisco's measured 400-450 LOC/hour threshold. These differ by roughly 10x. Both can be partly true only if the 13 agent reviewers genuinely substitute for the inspection the human is no longer doing — i.e. if the human has stopped verifying the artifact and started verifying a summary of a verification. The audit data reframes this from an alarming shortcut into the normal structure of professional sign-off. That is exactly what an engagement quality review is: AS 1220 has the reviewer evaluate the engagement team's significant judgments and explicitly not re-perform the audit, at 0.6-1.8% of engagement hours, with full liability attached. Verifying a verification is not the anomaly — it is how every high-throughput liability regime works. The difference is that audit has four tiers of filtering beneath the reviewer and a defined escalation path, and agent pipelines currently have neither. So the Every architecture is structurally sound and structurally incomplete, and it is still a self-report with no defect-escape measurement attached. Cisco's constant ~13 defects/hour discovery rate is the specific reason to doubt that a 10x larger artifact got 10x the inspection.
- The radiology AI result was the strongest counter-case, and verification has largely dissolved it. MASAI's 44.3% "workload reduction" is a count of readings, not clock time — the trial measured no reading times at all — and its detection difference was p=0.052, not significant. PRAIM's reading-time figure is a within-arm easy-versus-hard comparison, and its 56.7% is an explicitly fictitious automation simulation. So the headline evidence that machine assistance raises a verification ratio is not a measured time saving in either trial. Even taken at face value, both are AI reading the same homogeneous unit the human is trained on, inside a programme with a known prevalence base rate, continuous per-reader performance audit (the UK PERFORMS scheme), and an outcome checkable against cancer registries years later. None of those four conditions exist for agent-produced analytical deliverables. Citing MASAI as evidence that agent output is cheap to verify remains the single most available error in this domain — and it is now a worse error than the brief originally thought, because the number being cited is not the number people think it is.
- New contradiction, and it is the most uncomfortable finding in this pass: more verification volume does not buy more detection. Mammography readers at 2,000 / 5,000 / 10,000 cases per year held cancer detection flat at 7.6 / 7.6 / 7.7 per 1,000 while their recall rate fell 5.8 → 4.5 per 100. Five times the reps bought specificity, not sensitivity. And the automation-bias trial shows the reviewer's accuracy collapsing to 45.5% even among the most experienced readers when the machine was confidently wrong. Put together: a high-volume reviewer of machine output gets better at not crying wolf and no better at catching what matters, while remaining susceptible to exactly the failure mode agent output specializes in producing. That is the opposite of the reassuring story about experienced reviewers scaling.
Synthesis for RDCO
The honest answer is that this question cannot be answered as asked, and that is a finding rather than a failure — and the verification pass made it stronger, not weaker. There is no verification ratio. There is a family of ratios set by the shape of the unit being verified, and in measured reviewer-time per unit they span roughly four thousand-fold (0.98 minutes for a screening mammogram, 66 hours for an audit engagement quality review). The sharpest support is that the single most-cited rate in this entire literature comes from the study that measured that the rate is not a constant: Cisco's best fit for a reviewer's inspection rate was R²=0.29, with rates varying "widely, even with similar authors, reviewers, files, and size of the review." Anyone quoting a verification ratio is quoting the headline of a paper whose finding was that there isn't one.
One correction to this brief's own central claim, stated plainly rather than softened. The brief originally argued that professional-services deliverables resemble audit partner sign-off, "the slowest analogue on the board." The measured audit data does not support "slowest." Engagement quality review runs 66 hours against 11,277 total engagement hours — 0.6% in the US, 1.8% in Korea — and AS 1220 achieves that by having the reviewer evaluate the engagement team's significant judgments and explicitly not re-perform the audit, with four tiers of filtering (team sampling → staff review → manager review → partner review) between the underlying transactions and the reviewer's desk. Liability-grade sign-off is cheap as a share of the work product, precisely because it is not comprehensive. Any framing in which one reviewer reads one deliverable end-to-end overstates audit's actual coverage by orders of magnitude — including the framing this brief used.
The claim that survives is better than the one it replaces. The structural resemblance to audit holds: professional-services deliverables are heterogeneous, one-off, have no base rate, no compiler, and no registry to check against later. But what audit teaches is not "expect to be slow." It is that high-throughput liability-grade sign-off is achieved by sampling the judgment layer rather than verifying the artifact — and that this is only defensible when the tiers beneath the reviewer, and the escalation path out of it, actually exist. That is the fourth checkability mechanism, alongside unit homogeneity, a trained error prior, and a machine pre-filter. If a number has to be stated, state it in audit's shape rather than audit's speed: a senior professional can sign off a meaningful number of liability-bearing work products per week only to the extent that something underneath has already filtered them, and the review is scoped to judgments rather than to the artifact — and state that RDCO has not measured its own rate and no published source gives it.
This disconfirms the parent brief, and that should be written down rather than softened. [[2026-07-27-psf-pyramid-economics-post-agent-analyst-tier]] argued that the scarce layer moved from execution to verification, and treated that as establishing a new, higher leverage ceiling. The first half holds — the vault, the labor-market data, and the lateral-hiring shift all support it. The second half does not survive this pass. Agent leverage only converts into a higher ceiling if per-unit verification gets cheaper, and the only measurements available say it gets more expensive per unit while getting much cheaper per unit generated. That is a worse position than the parent brief implied: generation goes up 741%, releases go up 20%, and the reviewer's per-unit cost goes up. The leverage ceiling may be lower after agents than the parent brief assumed, not higher. Our confidence that "verification is the new ceiling" should stay high; our confidence that we know where that ceiling sits should drop to near zero, and nothing downstream should be priced or promised against it.
The mechanism that actually moves the ratio is the one all three analogues share, and it is not agent count. In every analogue where the ratio is high, something upstream has made the unit cheap to check: tests and CI for code, a homogeneous image type plus a trained prior plus per-reader audit for mammography, tiered staff review plus judgment-scoped sampling for audit. The lever is manufacturing checkability, not adding reviewers or adding agents. Three measured findings sharpen this into operating rules. (1) Batching buys nothing. Cisco's defect discovery rate is roughly constant at ~13/hour regardless of artifact size, so a bigger review package yields the same findings and more escapes — throughput scales with reviewer-hours, never with artifact volume. (2) Terseness is a throughput feature. Machine output in the second-reader slot costs +33 to +41 seconds per case; in the first-pass slot it saves 77-104 seconds; CAD was faster only when it emitted ≤20 marks. Put the agent before the human as a filter, not beside the human as a second opinion, and make it brief. (3) Self-annotation is the cheapest lever available. Cisco reviews with author annotations never exceeded 30 defects/kLOC versus up to ~130 without — correlational, but the agent analogue is exact: an agent that flags its own uncertainty restores the triage signal that uniformly-confident output destroys. And one hard constraint on all of it: the automation-bias trial measured the most experienced radiologists dropping to 45.5% accuracy when the machine was confidently wrong. Reviewer seniority does not protect against a confident wrong machine, so no amount of "a senior will catch it" substitutes for a mechanical gate. [[2026-06-03-parallel-agent-fde-capacity]] reached the same conclusion from the other direction — the way to raise N is to collapse per-client review cost so the human reviews exceptions and rejections rather than artifacts — and this brief's analogue spread is independent support for it. Concretely, for RDCO's own stack: the value of the verify-* fresh-eyes critic layer is not that it reviews things, it is that it converts a heterogeneous artifact into a unit with a pass/fail contract, which is what moves the ratio from the audit regime toward the code regime.
For the founder's live positioning — credibility in agents in production, feeding phData sales — the asset here is the measurement method, not the number. The verified negative result is the opportunity: every buyer deploying agents in production needs this number to staff and price against, no professional-services firm has published it, and the available substitutes are vendor telemetry from companies selling review tools. That is an unusually clean gap for the exact niche the founder has chosen. The credible contribution is not "the verification ratio is N" — we would be inventing it, and feedback_calibrate_overconfidence applies directly. The credible contribution is publishing the instrumentation: define the unit, define what sign-off means, log per-artifact review minutes and defect escapes on real work, and report the measured rate with its degradation curve. That is a "here is how we measured it on a live deployment" artifact, which is the shape of content that builds the agents-in-production credibility the founder is actually building toward, and it is ours to produce because we run the agent stack daily and nobody else is logging it. Four cautions, all of which this verification pass earned the hard way. Do not cite MASAI's 44% as a workload time saving — it is a count of readings and the trial measured no times at all. Do not cite PRAIM's reading-time numbers as AI-versus-unaided; the paper says the control arm was never timed. Do not restate Every's 15-minute figure as a throughput benchmark. And do not cite METR's 19% slowdown as settled — METR itself withdrew confidence in February 2026, and a technical reader will know that. The defensible version of each is the structural point underneath it, not the percentage on top of it.
Why this is in the vault
It closes the explicit open follow-up left in [[2026-07-27-psf-pyramid-economics-post-agent-analyst-tier]] ("what is the actual verification ratio... this is the number the whole new-leverage-ceiling argument depends on") with a disconfirming answer: the number does not exist, the published analogues differ roughly four thousand-fold in reviewer-time per unit because the ratio is a property of the unit rather than the reviewer, and the analogue nearest to professional-services work products reaches its throughput by scoping review to judgments rather than to artifacts. It therefore retires the parent brief's leverage-ceiling claim as unsupported, confirms [[2026-06-03-parallel-agent-fde-capacity]]'s N=2-3 as the operative planning number, and converts the gap into a specific credibility artifact for the founder's agents-in-production positioning: publish the instrumentation, not a borrowed figure.
Open follow-ups
Promote this brief's numbers from RECALLED to verified.RESOLVED 2026-10-08 by the three analogue researchers plus a targeted verification pass. All three named lookups are done: there is no PCAOB rulemaking ex-ante estimate, but PCAOB's 2018 post-implementation white paper gives measured EQ-reviewer hours (66/engagement, ~1% of total audit hours); the Google paper's distributions are retrieved (median 4 changes reviewed per developer-week, median 24 lines, median 1 reviewer); and NHSBSP Publication No 59 §3.4(d) is retrieved, with the 5,000-case figure confirmed as a competence floor and no rate cap existing anywhere in the guidance. Four figures were corrected rather than confirmed. What remains recalled is listed in the ⚠️ block.- Pull NBER w35275 ("Writing Code vs. Shipping Code") directly into the vault. It is currently held only as an AlphaSignal relay, and it is the single most load-bearing quantity we have on the size of the verification choke.
- PARTLY RESOLVED 2026-10-08. Does any analogue publish a degradation curve rather than a point rate — defect-escape rate as a function of review rate? Two now do, and both point the same way. Cisco publishes defect density against inspection rate (above average below 400 LOC/hr; below average in 87% of cases above 450) and the far more interesting result that discovery rate is flat at ~13/hour regardless of artifact size. Mammography publishes a volume-to-quality curve (PPV 12.9% → 14.4% → 17.0% at 2,000 / 5,000 / 10,000 cases per year) in which detection is flat and only specificity improves. Still open: the PERFORMS per-reader audit series itself, and any curve for an analytical (non-homogeneous) artifact.
- What is RDCO's own measured per-artifact review time and defect-escape rate across the
verify-*critic layer? We have months of real dispatch logs and have never computed it. This is the instrumentation the synthesis proposes selling, and we should measure ourselves first. - CONFIRMED OPEN — and the gap is wider than this brief assumed. Is there measured evidence either way on whether reviewing uniformly-confident agent prose is slower than reviewing human prose with legible hedging? Verified negative: no study holds artifact size constant and compares reviewer-minutes on human- versus AI-written work, for prose or for code. The code-review literature does not in fact answer this for code, which this brief previously implied it did. The closest measured proxies are the radiology second-reader-slot time penalties (+33 to +41 s) and the CAD-marks threshold (faster only at ≤20 marks). This remains the crux of the whole argument and is now the single most valuable thing RDCO could measure itself.
Does the Korean audit-hour disclosure regime yield a published hours-per-engagement series?RESOLVED 2026-10-08. Yes. Korea mandates EQR-hour disclosure and the series exists: EQCR hours = 1.8% of total audit hours (mean), 1.5% median, range 0.3-6.6%, N=4,580 firm-years FY2014-2016, against total audit hours of 715 mean / 651 median. It did not convert into a per-partner-week sign-off rate, because no jurisdiction publishes engagements per signing partner — the FRC-derived 0.68 PIE audits per Responsible Individual is a floor, not a count. Still open: whether any regulator publishes a true clients-per-partner denominator.- NEW, opened by this pass: does the four-tier filtering structure beneath an audit reviewer have a buildable agent equivalent? The audit data says liability-grade sign-off is cheap (0.6-1.8% of engagement hours) only because team sampling, staff review, manager review and partner review sit underneath it, with a defined escalation path. RDCO's
verify-*layer is one tier, not four, and has no escalation path. Specifying the missing tiers is the concrete design question this brief leaves behind. - NEW: no radiologist-performance-after-AI-withdrawal study exists (verified negative). The only measured deskilling signal is endoscopic (adenoma detection 28.4% → 22.4% post-CADe, observational, ~19 endoscopists). If verification capacity is the ceiling, reviewer skill decay under sustained agent assistance is the mechanism that would lower it over time, and nobody has measured it in any domain close to ours.
Related
- [[2026-07-27-psf-pyramid-economics-post-agent-analyst-tier]] — the parent brief this one partially disconfirms
- [[2026-10-08-finding-level-citation-audit-arxiv-gloss]] — the companion citation-audit brief; its 2-in-10 finding-level defect rate is what this brief's four corrections look like in practice
- [[2026-06-03-parallel-agent-fde-capacity]] — the vault's N=2-3 finding, independently supported here
- [[2026-06-07-alphasignal-async-agents-release-bottleneck]] — NBER w35275, the 741%-generation / 20%-release attenuation
- [[2026-01-23-every-ai-code-reviews]] — the 1,000-LOC-in-15-minutes counter-claim
- [[2025-10-27-every-six-engineers-ai-workflows]] — review bandwidth as the designed-around constraint
- [[2026-06-13-managed-ai-data-services-pricing-models]] — the fixed-backbone pricing stance that does not depend on this number
- [[2026-06-07-solo-fde-contract-structures]] — same
- [[2026-05-20-services-pricing-model-for-rdco-future]] — throughput capacity as prerequisite to the pricing model
- [[2026-05-13-fde-asymmetric-edge-rdco-positioning]] — the judgment-and-verification-priced artifact band
- [[2026-05-03-heyrico-service-as-a-software-shift]] — the price-compression gravity on anything priced as "faster"
Sources
Vault
- ~/rdco-vault/06-reference/research/2026-07-27-psf-pyramid-economics-post-agent-analyst-tier.md
- ~/rdco-vault/06-reference/research/2026-06-03-parallel-agent-fde-capacity.md
- ~/rdco-vault/06-reference/2026-06-07-alphasignal-async-agents-release-bottleneck.md
- ~/rdco-vault/06-reference/2026-01-23-every-ai-code-reviews.md
- ~/rdco-vault/06-reference/2025-10-27-every-six-engineers-ai-workflows.md
- ~/rdco-vault/06-reference/research/2026-06-13-managed-ai-data-services-pricing-models.md
- ~/rdco-vault/06-reference/research/2026-06-07-solo-fde-contract-structures.md
- ~/rdco-vault/06-reference/concepts/2026-05-20-services-pricing-model-for-rdco-future.md
- ~/rdco-vault/06-reference/concepts/2026-05-13-fde-asymmetric-edge-rdco-positioning.md
- ~/rdco-vault/06-reference/2026-05-03-heyrico-service-as-a-software-shift.md
Session-verified (web) — retrieved and read 2026-10-08
Code review
- Sadowski C, Söderberg E, Church L, Sipko M, Bacchelli A. "Modern Code Review: A Case Study at Google." ICSE-SEIP 2018, Gothenburg. DOI 10.1145/3183519.3183525. PDF: https://sback.it/publications/icse2018seip.pdf — median 4 changes reviewed per developer-week (80% fewer than 10); median ~3 authored/week; median 24 lines modified; median 1 reviewer; <25% of changes have >1 reviewer; median end-to-end latency <4 hours (wall-clock, not effort); ~9M changes, >25,000 authors and reviewers, Jan 2014-Jul 2016
- Cohen J. "The Largest Case Study of Code Reviews — Ever." StickyMinds, 31 October 2006. https://www.stickyminds.com/article/largest-case-study-code-reviews-ever — Cisco Systems, 10 months, 2,500 reviews, 3.2M LOC, 50 developers on three continents; recommendations: <200-400 LOC per review, inspection rate <300-500 LOC/hour, <60 minutes per review (60-90 min = general concentration dropoff); ~15 defects/hour. Retrieved via Playwright; the site 403s both WebFetch and curl
- Cohen J. "Code Review at Cisco Systems," ch. 5 of Best Kept Secrets of Peer Code Review, Smart Bear Software, 2006. https://static0.smartbear.co/support/media/resources/cc/book/code-review-cisco-case-study.pdf — measured thresholds: reviewers slower than 400 lines/hour above average at finding defects, above 450 lines/hour defect density below average in 87% of cases; discovery rate ~13/hour with 94% of reviews under 20/hour; inspection-rate best fit R²=0.29 with rates varying "widely"; author-annotated reviews never exceeded 30 defects/kLOC vs up to ~130 unannotated (correlational). Conflict of interest: Cohen founded SmartBear, which sells the review tool used
- SmartBear. "Best Practices for Code Review." https://smartbear.com/learn/code-review/best-practices-for-peer-code-review/ — source of the widely-restated "under 500 LOC per hour" attributed to unspecified "SmartBear research," and of the uncited "200-400 LOC over 60-90 minutes yields 70-90% defect discovery." Vendor marketing, undated, not the Cisco data. Do not cite
- METR. "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity," 10 July 2025. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/ (paper: arXiv:2507.09089) — RCT, 16 developers, 246 issues randomized to allow/disallow AI; tasks took 19% longer (CI +2% to +39%); developers forecast 24% speedup and estimated 20% speedup afterwards
- METR. "We are Changing our Developer Productivity Experiment Design," 24 February 2026. https://metr.org/blog/2026-02-24-uplift-update/ — 57 developers, 143 repos, 800+ tasks; returning-developer subset -18% (CI -38% to +9%); METR calls its own data "an unreliable signal of the current productivity effect of AI tools" and "only very weak evidence"; cause = developers declining to work without AI plus a pay cut from $150 to $50/hr
Audit
- PCAOB. AS 1220: Engagement Quality Review. https://pcaobus.org/oversight/standards/auditing-standards/details/AS1220 — adopted as AS No. 7 via PCAOB Release No. 2009-004; amendments to .04/.05/.10/.15 effective 2026-12-15 (Release No. 2024-005). EQR + concurring approval required on every issuer audit (.01); objective is evaluation of the engagement team's significant judgments (.02, .09, .10); reviewer must not make decisions for or assume responsibilities of the engagement team (.07); partner-or-equivalent, may be external (.03); two-year cooling off from engagement partner (.08); approval only if "not aware of a significant engagement deficiency" (.12); report may not be released before approval (.13). Retrieved via Playwright; pcaobus.org 403s WebFetch and curl
- Gurbutt J, McAdams M, Shih I. "Staff White Paper on Post-Implementation Review of AS 1220, Engagement Quality Review." PCAOB Office of Economic and Risk Analysis, December 2018, 116 pp. https://assets.pcaobus.org/pcaob-dev/docs/default-source/economicandriskanalysis/pir/documents/pir-as1220-staff-white-paper.pdf — "the average reviewer hours per engagement is 66 (68) hours, comprising approximately 1 percent of total audit hours" (p.18); by issuer market cap <$700M → 53 hrs, $700M-$5B → 67, >$5B → 103; average total audit hours per engagement 11,277 (US GNF), 824 (triennially-inspected non-affiliated). Retrieval note:
assets.pcaobus.orgPDFs download fine via curl even thoughpcaobus.org403s - Song H, Kim J (2021). Journal of Business Economics and Management 22(5):1269-1287. DOI 10.3846/jbem.2021.14934 — Korean EQCR hours = 1.8% of total audit hours (mean), 1.5% median, range 0.3-6.6%, N=4,580 firm-years FY2014-2016
- Behrend J, Khan I, Ko Y, Park Y (2020). Journal of International Accounting Research 19(3) — Korean total audit hours per engagement 715 mean / 651 median
- Financial Reporting Council. Key Facts and Trends in the Accountancy Profession, 23rd ed., 2025, Figure 33 (workbook parsed) — Big Four UK 2024: 883 PIE audit clients across 1,305 Responsible Individuals = 0.68 PIE audits per RI (1.52 per audit principal); average audit fee income per RI £2.31m (Big Four £2.82m). PIE clients only — a floor, not a count
- Suzuki K, Takada T (2024). Journal of Business Finance & Accounting 51(5-6):1046-1083 — 89% of Japanese listed-firm audits have a lead partner with at least one other client, N=57,513 firm-years
Radiology
- NHS Breast Screening Programme. Publication No 59, Quality Assurance Guidelines for Breast Cancer Screening Radiology, 2nd ed., March 2011 (updated Dec 2018). https://associationofbreastsurgery.org.uk/media/imafaa5n/quality_assurance_guidelines_for_breast_cancer_screening_radiology_updated_dec_2018.pdf — §3.4(d) readers must "undertake a minimum of 5000 screening and/or symptomatic cases per year"; §3.5.2 double reading "recommended and should be considered mandatory in units that have moved entirely to digital mammography." Verified negative: no cases-per-hour or per-session cap anywhere in the guidance (full 1,727-line extracted text grepped)
- Partridge GJW et al. "How long does it take to read a mammogram?" European Journal of Radiology 2024;177:111535. PMID 38852330 — 48 readers, 34,210 FFDM cases, PROSPECTS trial within the NHSBSP: 2D FFDM 0.98 ± 0.30 min/case (~61/hour); DBT 2.09 ± 0.64 min/case (~29/hour), p<0.001; radiologists vs radiographers 2.06 vs 2.14 min, p=0.71; experience buys ~22%, plateaus inside 9 months
- Cornford E et al. European Radiology 2021;31(9):6909-6915. PMID 33630161 — 759 English readers, volumes 1,000-45,000 cases/year; at 2,000 / 5,000 / 10,000 cases/year: PPV 12.9% → 14.4% → 17.0%, recall 5.8 → 5.3 → 4.5 per 100, cancer detection rate flat at 7.6 / 7.6 / 7.7 per 1,000
- Lång K et al. (MASAI). Lancet Oncology 2023;24(8):936-944. PMID 37541274 — the 44.3% "workload reduction" is a count of screen readings (46,345 vs 83,231), not clock time; no reading times were measured; CDR 6.1 vs 5.1 per 1,000 at p=0.052, not significant
- Künzel/Eisemann et al. (PRAIM). Nature Medicine 2025. DOI 10.1038/s41591-024-03408-6 — reading time "was measured in the AI group only as it was technically not possible to measure this in the control group"; the 39s-vs-67s figure is within-arm easy-vs-hard, not AI-vs-unaided; the 56.7% headline is an explicitly "fictitious AI automation scenario" equal to the triage fraction (262,055 of 461,818 exams)
- Dratsch T et al. "Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance." Radiology 2023;307(4):e222176. PMID 37129490. https://research.vu.nl/ws/portalfiles/portal/291201748/Dratsch_et_al._2023_-_Automation_Bias_in_Mammography_-_The_Impact_of_Artificial_Intelligence_BI-RADS_Suggestions_on_Reader_Performance.pdf — 27 radiologists, staged incorrect AI BI-RADS: accuracy correct-vs-incorrect AI 79.7% → 19.8% (inexperienced), 81.3% → 24.8% (moderate), 82.3% → 45.5% (very experienced), all p≤.003
- Nair A et al. European Radiology 2018;28(1):226-234. PMID 28643093 — lung-screening LDCT 9.1 ± 2.3 min/case independent read, 7.2 ± 1.0 min with second reader; sensitivity 77.5% → 90.8%; false positives per case 0.33 → 0.60
- Budzyń K et al. Lancet Gastroenterology & Hepatology 2025. PMID 40816301 — unaided adenoma detection 28.4% → 22.4% after CADe adoption; observational, ~19 endoscopists, 4 Polish centres; adenomas-per-colonoscopy unchanged. Striking signal, not causation
Verified negatives
- Extended web search for a published per-reviewer weekly capacity figure for agent output — returned no such figure. 2026-10-08
- No study holds artifact size constant and compares reviewer-minutes on human-written versus AI-written work, for code or for prose. 2026-10-08
- No radiologist-performance-after-AI-withdrawal study exists. 2026-10-08
- No PCAOB rulemaking ex-ante estimate of EQR hours per engagement exists; the 2018 post-implementation white paper is the only verifiable PCAOB hours figure. 2026-10-08
- No published mean of engagements per signing audit partner exists in any jurisdiction. 2026-10-08
Search-relayed — vendor-published, not fetched, commercial interest in review tooling
- LinearB, ~8.1M pull-request analysis — AI-change review-start delay (~4.6x) and acceptance rate (~32.7% vs ~84.4%)
- Faros AI — ~98% more PRs merged against ~91% higher review time; a summary reporting median review time +441.5% vs task throughput +33.7%
- Digital Applied developer survey, n=2,847, Q1 2026 — median 11.4 hours/week reviewing AI-generated code
Recalled, named for checking — still NOT retrieved
- Michael Fagan, IBM software inspection (1976) — the ~125-150 LOC/hour recommended rate. Paywalled; could not be tied to any rate in the original text, and the circulating figures disagree (125 vs 130-150), with one source attributing 100-150 to Radice rather than Fagan. Treat as folklore
- Goodwin & Wu, audit-partner busyness (Australian partner-signature data); López & Peters, busy-season workload compression, Auditing: A Journal of Practice & Theory. Both paywalled
- McDonald et al., Mayo Clinic, Academic Radiology (~2015) — cross-sectional imaging workload; one image every 3-4 seconds
- NBER working paper w35275, "Writing Code vs. Shipping Code" — held only as an AlphaSignal relay via [[2026-06-07-alphasignal-async-agents-release-bottleneck]]; still never pulled directly
Peer-reviewed secondhand — quoted inside a primary source, not opened directly
- Rigby & Bird, FSE 2013 — comparison change sizes (AMD 44 lines, Lucent 263 lines, Microsoft between) and times to approval. Paywalled; available only as quoted inside the Google ICSE-SEIP paper
Search-relayed — weak, directional only
- AI-as-second-reader time penalties: lung-nodule CT +33 to +41 s/case as second reader vs 77-104 s saved as first-pass reader (PMC10682324); CT colonography CAD-as-first-reader 243 → 239 s, faster only at ≤20 CAD marks; chest X-ray pulmonologists 67 → 30 s, radiologists unchanged at 61 s (PMID 37750179)
- ~45% of merged Claude Code-authored PRs required further human modification (567 PRs, 157 projects, arXiv 2509.14745); ~10% of agent-generated methods deleted during review; GitClear duplicated code blocks up 8x in 2024 (vendor-published, their own customers' repos)
Explicitly rejected — do not use
- "61% of AI-agent PRs received no human review," attributed to a 2026 peer-reviewed study — could not be traced past secondary reporting
- SmartBear's "200-400 LOC over 60-90 minutes yields 70-90% defect discovery" — no source cited anywhere; the figure most often laundered into blogs as a Cisco research finding
- "500 LOC/hour" as the Cisco study's measured threshold — it is SmartBear vendor copy citing unspecified "SmartBear research." The measured pair is 400 / 450
- GitClear's "code churn 3.3% → 7.1%" — year labels do not reconcile with GitClear's own dataset
- Finnish "3 to 6 public audit clients" busyness figure, and the 2,283-partner Form AP sample — unreachable, could not be verified
- MASAI's 44.3% as a reading-time saving, and PRAIM's 39s-vs-67s as AI-assisted-vs-unaided — both are misreadings of what those trials measured