METR's Anthropic embed, read at the source: a ~2x benchmark-index speedup with a wide band and redacted evidence - not a measured 10x, and not a superintelligence forecast
The question
What did METR's three-month embed inside Anthropic actually measure and report in its own publication (task set, baseline, the 'well-scoped tasks' qualifier, uncertainty), and does it support the ~10x AI-research uplift and 'superintelligence within a year' readings that Dwarkesh and Innermost Loop build on it?
Context: the vault held only second-hand paraphrase of this report across at least six notes in two weeks. This brief is the first primary-source read.
Read-quality note. METR's page and Ajeya Cotra's post were read through a fetch-and-extract pass, not eyeballed line by line. Quotes below are the ones the extractor returned as verbatim; numbers marked (extract) are paraphrased from the page. Treat exact wording as high-confidence, not certified. Anyone quoting this publicly should open the METR page first.
What we already know (from the vault)
- [[2026-11-13-dwarkesh-recursive-self-improvement-has-started]] and its transcript carry the most-repeated version: METR "spent three months embedded inside of Anthropic" and concluded progress "on well-scoped tasks is 2x what it was a year ago," with a majority attributed to AI doing R&D. Dwarkesh then says 2027 is "the straightforward implication" of putting that into a takeoff model, and quotes Cotra's "extreme superintelligence within a year" line. He also hedges: his own median is not 2027, and he gives 25%+ that the milestone lands after 2032.
- [[2026-11-09-innermost-loop-dreaming-memory-rsi-campaign-issue]] compresses it to: "METR says Anthropic's progress doubled; Ajeya Cotra reads it as 10x uplift and superintelligence within a year," bundled with time horizons doubling every two months. Our own note already flagged that "within a year" is Cotra's extrapolation, not METR's.
- [[2026-11-06-alphasignal-gpt6-cyber-metr-capability-doubling]] and [[2026-11-06-innermost-loop-post-midterm-moratorium-rsi-sandbagging]] had the most accurate numbers second-hand: 11 weeks, ~2x the March 2024 to October 2025 rate, 1.3x-3x range, ~80% AI-attributed, staff reporting up to 12x, heavy redaction.
- [[2026-11-16-dwarkesh-ais-might-overthrow-the-government-within-a-year-paul-christiano]] has Christiano stressing the evidence-quality problem: the report is largely redacted in public form.
- [[2026-11-17-innermost-loop-huang-surrender-nvidia-put-san-francisco-declaration-rsi-by-march]] relays an unverified rumor that Anthropic expects RSI by March. That is a separate rumor; nothing in the METR report supports or mentions it.
What the web says
- The primary source exists and is public. METR, "Independent Assessment of AI R&D Acceleration at Anthropic in October 2026," published 2026-11-05 (contributors Tom Cunningham, Nate Rush, Parker Whitfill, Hjalmar Wijk). The engagement ran August 12 to October 29, 2026: about 11 weeks, 2.5 FTE, extended from an original 4 weeks (extract). "Three months embedded" is a rounding. METR saw internal benchmark scores, algorithmic-efficiency data, headcount, training compute, internal uplift surveys, inference spend, and about 15 staff interviews. It had no query access to unreleased models (extract). (METR)
- Headline, in METR's words: "Anthropic is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025." Two corrections to the circulating version: the term is "well-specified," not "well-scoped," and the baseline is a 19-month historical trend, not "a year ago."
- Task set = a benchmark index, not research output. The central measure is AECI, Anthropic's internal version of Epoch's Capabilities Index, built from 15 benchmarks whose names are redacted (grouped as math, software engineering, science, multimodal, language/other). METR describes it as measuring "performance on well-specified tasks that are easy to grade automatically or with a language model." The trend moved from roughly +15 index points per year to roughly +29 after a fitted breakpoint of October 7, 2025 (extract). So "2x" means the slope of a benchmark composite doubled. It is not a measurement of researcher productivity or of research taste.
- Uncertainty is wide and METR says so. Point estimate 1.9x, 95% CI 1.5x-2.8x; "We would not be surprised if the current rate of growth was closer to 1.3x or 3x the historical rate." Across alternative link functions the result ranges 1.1x to 2.9x (extract). The same method gives about 1.4x for non-Anthropic frontier models (extract), which implies part of the speedup is industry-wide rather than specific to Anthropic's internal AI use. On scope: "We have additional uncertainty in how this acceleration translates to broader and less well-specified tasks."
- AI attribution is a guess, labeled as one. "We guess that a majority (perhaps ≈80%) of the current (October 2026) acceleration is due to AI." METR also writes: "It's possible that the non-AI factors explain all or most of the recent acceleration." Of 11 researchers interviewed on whether AI is driving acceleration: 2 yes, 3 lean yes, 4 unclear, 2 no (extract).
- The 12x number is a Slack poll. Staff self-reported uplift rose from about 1.2x (January 2025) to about 12x (October 2026). METR: "We consider this to be only very loose evidence." Sample size is not stated (extract). Calibration reminder from our own files: METR's February 2026 developer study found self-report an unreliable guide to measured effect.
- Redaction is the load-bearing caveat. Anthropic held unilateral redaction rights and removed most of the evidence METR found most informative; METR says "the public has to rely on METR's judgment much more than would be ideal." Anthropic also cancelled a compute-allocation verification midway. METR found no significant bottlenecks over the next 12 months but says its view changed several times and it is not confident (extract). The report makes no superintelligence or timeline claim; it flags "significant risk" of further compounding.
- Where 10x and "within a year" come from: Ajeya Cotra's blog, not METR. Her post (Planned Obsolescence, cross-posted Nov 6) opens: "I did not work on this project or see the unredacted report" and "All opinions are my own." The ~10x is derived, not observed: a "very simple Cobb-Douglas model where experiment compute and labor are similarly important" turns a ~2-3x progress multiplier into "~4-9x speed up in labor," then "maybe the necessary labor uplift is ~10x," which she calls "very uncertain" and "unlikely to be uniform." The famous sentence is doubly conditional: "If we take it at face value... and assume acceleration from AI is smooth, this naively implies that we would reach extreme superintelligence within a year." She declines to give a bottom-line forecast. (Cotra via EA Forum)
Convergences and contradictions
- Agree: the 2x headline, the ~80% attribution, the 1.3x-3x band and the redaction all check out against the primary. The AlphaSignal and Nov 6 Innermost Loop notes were accurate. Dwarkesh quotes Cotra correctly and keeps her conditionals.
- Disagree / drift: (1) "well-scoped" and "a year ago" are Dwarkesh's paraphrases; the source says "well-specified" against a March 2024 to October 2025 baseline. (2) The Nov 9 Innermost Loop line drops Cotra's "naively" and both conditionals, turning a thought experiment into a reading. (3) Our Sept 1 note describes Cotra as a METR researcher, which gives the 10x figure borrowed institutional weight it does not have; she disclaims involvement. (4) "Time horizons doubling every two months" is not in the METR report or in Cotra's post as extracted. Its source is unverified and it should not be cited as METR's finding.
- Not found: no independent skeptic piece surfaced in the search budget. The strongest critique available is the report's own limitations section.
Synthesis for RDCO
Graded against the primary, the three claims land differently. "Anthropic's progress roughly doubled, mostly from its own AI" is supported as METR's judgment, with medium confidence in the 2x and lower confidence in the "mostly from AI" part, which METR itself calls a guess and which a third of its interviewees did not endorse. "~10x research uplift" is not supported as a finding. It is the product of one outside commentator's two-factor model plus an unsized Slack poll that METR calls very loose evidence. "Superintelligence within a year" is not supported at all as a forecast; it is a naive limit case its own author flags as naive. Anyone citing METR for it is citing the wrong document.
The practical reading for the L5 plan: the report is real evidence that the capability curve RDCO rides on Claude is steepening on tasks that can be graded automatically. That is precisely the kind of work the agent fleet does (well-specified, verifiable, gate-checked), so the relevant half of the finding transfers to us better than it transfers to "automated AI researcher" claims. What it does not license is planning around a discontinuity. A sober planning assumption is "frontier model generations arrive about twice as fast as the 2024-25 cadence, plausibly 1.3x to 3x," which shortens the shelf life of any harness scaffolding and raises the value of the generation-survival work already in the vault. It does not justify treating 2027 as a wall.
The vault-hygiene finding matters as much as the grading. In two weeks one report fanned out into six-plus notes, and the strongest claims attached themselves to the most authoritative name. The pattern to watch: a hedged primary (METR) gets a conditional gloss (Cotra), which gets a compressed headline (Innermost Loop), which gets logged as cross-confirmation. Three newsletters repeating one redacted report is one data point, not three. Our Nov 9 note counted it as "one more mark" toward the L5 runway question; that mark should be read as a repeat, not a confirmation.
On the two meanings of "recursive self-improvement": this report is strictly meaning one (frontier-lab AI R&D speedup, measured as a benchmark-index slope). It says nothing about meaning two (practitioner generate-evaluate-rewrite loops like Weco's outer-loop agent or our own improve cycle). The vault currently files both under one tag. A short concept doc separating them, with this brief as the anchor for meaning one, would stop the 24 mentions from blurring into one thesis. For any Sanity Check or phData-facing use: the defensible sentence is "METR estimates roughly 2x, range 1.3x to 3x, on automatically gradable tasks, with most evidence redacted." Ray's pick, not a founder position.
Why this is in the vault
It replaces second-hand paraphrase with a source-checked grading for the specific claim that [[project_l5_north_star_strategic_direction]] leans on when sizing the L4-to-L5 runway against Anthropic's trajectory, and it gives a citable correction for the Nov 9 and Nov 13 notes that attribute the 10x and one-year figures upstream of where they actually originate.
Open follow-ups
- Did the 1.4x figure for non-Anthropic frontier models come from the same AECI method, and what does it imply about how much of the 2x is Anthropic-specific versus industry-wide?
- Where does "time horizons doubling every two months" originate (a METR time-horizon update, the AI Futures blog, or an analyst), and does that source hold up?
- What does the AI Futures "five months ahead of AI 2027" claim rest on, and is it independent of this same METR report?
- OpenAI now has four embedded assessors (Apollo, METR, Transluce, Goodfire). Will any publish a comparable acceleration estimate, giving a second lab's data point with different redaction terms?
- Has anyone outside the safety community published a methodological critique of AECI-slope as an R&D-acceleration measure (benchmark saturation, breakpoint fitting)?
- Vault: write the concept doc that splits "recursive self-improvement" into lab R&D speedup versus practitioner rewrite loops.
Related
- [[2026-11-13-dwarkesh-recursive-self-improvement-has-started]]
- [[2026-11-09-innermost-loop-dreaming-memory-rsi-campaign-issue]]
- [[2026-11-06-alphasignal-gpt6-cyber-metr-capability-doubling]]
- [[2026-11-06-innermost-loop-post-midterm-moratorium-rsi-sandbagging]]
- [[2026-11-16-dwarkesh-ais-might-overthrow-the-government-within-a-year-paul-christiano]]
- [[2026-11-17-innermost-loop-huang-surrender-nvidia-put-san-francisco-declaration-rsi-by-march]]
- [[2026-09-01-dwarkesh-ajeya-cotra-openai-agent-swarm]]
- [[2026-11-13-every-your-attention-is-ai-bottleneck]]
- [[2026-06-04-anthropic-institute-recursive-self-improvement]]
- [[2026-07-15-innermost-loop-recursive-self-improvement-chip-cycle]]
- [[2026-08-19-agent-scaffolding-generation-survival]]
- [[2026-10-08-verification-ratio-senior-signoff-throughput]]
- [[project_l5_north_star_strategic_direction]]
Sources
Web:
- METR, "Independent Assessment of AI R&D Acceleration at Anthropic in October 2026," 2026-11-05. https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment (primary; read via extraction)
- Ajeya Cotra, "AI is dramatically accelerating AI research right now," Planned Obsolescence, cross-posted 2026-11-06. https://forum.effectivealtruism.org/posts/g7DQdcpnFq65CcjP5/ai-is-dramatically-accelerating-ai-research-right-now (read via extraction)
- Search-result summaries only, not fetched: https://alphasignal.ai/news/metr-finds-anthropic-s-ai-assisted-r-d-moving-twice-as-fast ; https://alignmentforum.org/posts/N5AmYkRks5CqAxvBg/metr-independent-assessment-of-ai-r-and-d-acceleration-at ; https://blog.aifutures.org/p/q35-timelines-update-looks-like-the
Vault:
- 06-reference/2026-11-13-dwarkesh-recursive-self-improvement-has-started.md (and its transcript in 06-reference/transcripts/)
- 06-reference/2026-11-09-innermost-loop-dreaming-memory-rsi-campaign-issue.md
- 06-reference/2026-11-06-alphasignal-gpt6-cyber-metr-capability-doubling.md
- 06-reference/2026-11-06-innermost-loop-post-midterm-moratorium-rsi-sandbagging.md
- 06-reference/2026-11-16-dwarkesh-ais-might-overthrow-the-government-within-a-year-paul-christiano.md
- 06-reference/2026-11-17-innermost-loop-huang-surrender-nvidia-put-san-francisco-declaration-rsi-by-march.md
- 06-reference/2026-09-01-dwarkesh-ajeya-cotra-openai-agent-swarm.md
- 06-reference/2026-11-13-every-your-attention-is-ai-bottleneck.md
- 06-reference/2026-06-04-anthropic-institute-recursive-self-improvement.md
- 06-reference/2026-07-15-innermost-loop-recursive-self-improvement-chip-cycle.md
- 06-reference/research/2026-08-19-agent-scaffolding-generation-survival.md
- 06-reference/research/2026-10-08-verification-ratio-senior-signoff-throughput.md