Why this is in the vault
Strong mapping to the compounding-intelligence thesis. Jang's central argument adds a structural data-point to the optimizer-optimizing-optimizer cluster ([[2026-05-15-innermostloop-singularity-optimizing-optimizer]]): the LLM RLVR loop has a known sample-efficiency hole (credit assignment) that AlphaGo-style MCTS-per-step would close. If/when frontier labs ship MCTS-augmented training at LLM scale, expect a non-linear capability jump — that's a stage marker to watch for. Also delivers the "initialize from something that works" methodological principle that maps to Ray's build-cycle discipline, and the compression-of-simulation mental model for why pre-trained agents are productive on novel problems. Tracked-author promotion candidate (see note at bottom).
Episode summary
Eric Jang spent his recent sabbatical rebuilding AlphaGo from scratch using modern AI tooling, then released it as the autogo repo on his GitHub (evjang.com / github.com/ericjang). The walkthrough is structured as a worked example of "the primitives of intelligence" — search, learning from experience, self-play — but the load-bearing argument is comparative: AlphaGo's MCTS gives a strictly-better training target on every move, while LLM RLVR (naive policy-gradient over a 100k+ token trajectory) has to figure out backwards which tokens actually produced the right answer. Jang's framing is that the way humans learn is much closer to the AlphaGo case than the LLM case, and that we may be leaving large efficiency gains on the table by not running search-per-step in LLM training.
Cost data point Jang opens with: David Wu's open-source KataGo (2020) achieved a 40x compute reduction vs DeepMind's AlphaGo Zero. Combined with LLM coding agents in 2026, what took a research team millions of dollars can now be re-derived for a few thousand dollars of rented compute. That's the meta-thesis — AlphaGo as the cleanest cheap teaching artifact for compounding-intelligence primitives.
Key arguments / segments
[00:00:00] Why AlphaGo now? — Jang frames it as the cleanest worked example of intelligence primitives because Go is deterministic, fully observable, has a clean reward signal, and was previously "intractable for search yet solved by deep learning." KataGo's 40x compute reduction means a single researcher can reproduce it on desktop Blackwell GPUs.
[00:02:00–00:10:00] Go fundamentals + game-tree depth. Games run ~250–300 moves under Tromp-Taylor scoring, so search tree depth is large; the mystery is "how does a 10-layer network amortize simulation of something so deep in the game tree."
[00:15:00–00:30:00] MCTS, UCB, and exploration vs exploitation. Walks through Monte Carlo Tree Search, upper-confidence-bound action selection, and the formula adjustments to handle Go's much higher branching factor than standard bandit problems.
[00:35:00–00:50:00] Policy + value network training; expert-data bootstrap. Standard research wisdom: "Always initialize from something that works, then make it better; don't start from something that doesn't work and try to make it work." Bootstrap by predicting moves from games where an expert won, plus predict per-board win probability. Logits naturally converge to 0.5 on opening boards (half of branches lead to wins) and resolve toward 0/1 as the game progresses.
[01:10:00–01:15:00] Two-headed networks. AlphaGo Lee had separate policy + value networks; later papers merged into shared-trunk two-headed network. "Presumably saves compute" but rigorously answering by how much is non-trivial. The policy and value heads should agree at evaluation time — internal consistency check.
[01:15:00–01:25:00] AlphaGo as "compression of simulation." Jang's most striking framing: Go is chaotic (one stone placement disrupts the entire prediction), yet the value network reliably predicts the winner. Analogy to weather: we don't care about wind velocity at a specific lat/long, we care about macroscopic structure (where's the hurricane). Compares to Lorenz attractor — we may not know where on the attractor we'll land but we know the attractor's shape. AlphaGo was the first paper to "really show this profound level of simulation being compressed into a small amount of forward-pass computation." Contrast: a hash function is also initial-condition-sensitive but has no macro structure, so no equivalent value function exists.
[01:30:00–01:45:00] The credit-assignment problem in LLM RLVR. This is the central comparison and the most cite-able section. In RLVR you reward "trajectories that pass the unit test" and upweight all tokens in those trajectories — Karpathy's "supervision through a straw" critique. With AlphaGo, MCTS gives you a strictly better action target on every single move, sidestepping credit assignment entirely. Sample-efficiency gap follows directly. The advantage-estimation trick in RL (push up actions better than baseline, push down worse-than-baseline) is the workaround, but it's a workaround for not having search.
[01:45:00–01:50:00] Q-learning as backward propagation when forward search isn't available. Jang reframes Q-learning historically: we adopted it because we couldn't model dynamics with a world model. The two approaches (MCTS-forward vs Q-learning-backward) become "preferable" along the dimension of whether you can simulate forward — and LLMs are increasingly in the regime where forward simulation IS possible (sub-agents, sandbox execution).
[02:00:00–02:10:00] Practical training tricks. Initializing self-play against KataGo as opponent removes the need for KataGo's auxiliary supervision objectives. Training on 9x9 boards transfers cleanly to 19x19 and cuts the supervised-learning-baseline warm-start time (AlphaGo Zero spent first 30 hours just catching up to supervised baseline). Off-policy buffer works fine in AlphaGo despite the common "off-policy is bad" intuition.
[02:30:00–02:35:00] Google's bias toward old approaches. Discusses the meta-question of whether DeepMind's deep investment in game-tree-style approaches hobbled or helped them on LLMs. Jury still out — Gemini 3 caught up, but maybe the late start was because they were investing in TPU stack instead, which compounded later. "Hard for humans to reason about the optimal research strategy even with the data we have today."
[02:35:00–02:37:00] Resources. evjang.com (blog post linking to interactive tutorial), GitHub
ericjang/autogo(reproducible repo), "as rocks may think" blog post (Jang's broader thesis on thinking-as-a-primitive in computer science).
Notable claims
The LLM RLVR sample-efficiency penalty is structural, not incidental. Naive trajectory-level credit assignment is qualitatively worse than per-move search-derived targets. This is a falsifiable claim — anyone running MCTS-style per-step search in LLM training should see meaningful improvement on tasks where forward simulation is tractable.
The KataGo 40x compute multiplier + 2026 LLM coding agents = AlphaGo reproducible for ~$1k–$5k. Worth noting as a meta-data-point on the cost-collapse of historically expensive research artifacts.
AlphaGo as compression-of-simulation, not as search. Jang's reframing is that the network learns to predict the macroscopic structure of a chaotic system — making it a closer cousin to AlphaFold / weather-prediction / climate models than to game-tree search per se.
Initialize from something that works. Restated research-methodology principle that applies beyond AlphaGo: the warm-start always beats from-scratch in compute-bound regimes.
Guests
Eric Jang — Independent / sabbatical. Formerly VP AI at 1X Technologies (humanoid robotics), previously Senior Research Scientist at Google DeepMind Robotics. Blogger at evjang.com (long-running, technically dense). GitHub ericjang. Author of the "as rocks may think" essay on thinking-as-a-primitive. Not yet a tracked author in the vault — strong candidate for promotion based on this episode (clean reframings, original recent work, willingness to publish reproducible code).
Mapping against Ray Data Co
Strong mapping to the compounding-intelligence thesis. The Innermost Loop note ([[2026-05-15-innermostloop-singularity-optimizing-optimizer]]) named Poetiq Meta-System + Prime Intellect + Codex/Claude Code as the May-2026 SOTA stage markers for the optimizer-optimizing-optimizer loop. Jang's central argument adds a structural data-point: the LLM RLVR loop has a known sample-efficiency hole (credit assignment) that AlphaGo-style MCTS-per-step would close. If/when frontier labs ship MCTS-augmented training at LLM scale, expect a non-linear capability jump — that's a stage marker to watch for.
Secondary relevance:
- "Initialize from something that works" — methodological principle Ray applies on RDCO build cycles (don't greenfield, fork the closest-working artifact).
- The "compression of simulation" framing is a useful mental model for thinking about why Ray's own coding agent can be productive on novel problems despite never having seen them — pre-training is doing the compression.
No contradictions to existing vault positions. No re-frame opportunity for Sanity Check (this is engineering-deep; the credit-assignment story is one possible angle but better suited to a niche technical audience than the SC reader).
Related
- [[2026-05-15-innermostloop-singularity-optimizing-optimizer]] — primary stage-markers note; this episode adds the "credit assignment as the structural gap in LLM RL" data-point
- [[transcripts/2026-05-15-dwarkesh-eric-jang-alphago-from-scratch-transcript]] — full transcript
Tracked-author candidate
Eric Jang — recommend promoting to tracked author. Signals: (1) original technical reframing (compression-of-simulation, credit-assignment-as-structural-gap), (2) publishes reproducible code, (3) writes long-form essays at evjang.com (specifically "as rocks may think" cited by Jang himself as the broader thesis), (4) cross-discipline credibility (robotics + RL + game AI). Add to author whitelist for cross-check + graph ingestion.