"Dylan Patel — The single biggest bottleneck to scaling AI compute" — Dwarkesh Patel

Why this is in the vault

Dylan Patel runs the most cited semiconductor data operation in the AI industry; this is the definitive public explanation of why ASML EUV tools — not power, not data centers — are the binding constraint on how much AI compute the world can produce through 2030. The bottleneck progression framework and the 200 GW ceiling math are directly load-bearing for RDCO's capital cycle thesis and client advisory on compute access.

Episode summary

Patel walks Dwarkesh through the full AI compute supply chain from hyperscaler capex allocation through to ASML EUV tool production limits, arguing that the "single biggest bottleneck" has already shifted from power and data centers back to chip manufacturing. He quantifies the 2030 ceiling at roughly 200 gigawatts of deployable AI compute capacity, explains why memory is simultaneously crashing smartphone economics, and closes on why space data centers make no sense while chip scarcity persists.

Key arguments / segments

• [00:01:00] Hyperscaler capex anatomy — The $600B big-four capex figure is not all being spent on current-year hardware; a large chunk is turbine deposits for 2028-29, data center construction for 2027, and power purchasing agreements. Incremental US capacity this year is roughly 20 GW. Anthropic's conservative posture left them scrambling for last-minute compute at a markup while OpenAI's aggressive 5-year contract strategy locked in margin advantage.

• [00:12:00] GPU depreciation reversal — Michael Burr's "3-year GPU depreciation" thesis is wrong in a value sense: an H100 is worth more today than in 2022 because better models (GPT-5.4) run on it more efficiently and produce more tokens of higher quality than GPT-4 did. Value is determined by the utility you can extract today, not comparative specs of newer chips, as long as manufacturing is constrained.

• [00:21:00] Early commitment = permanent margin advantage — Companies that locked in 5-year contracts when capacity was cheaper have structurally lower compute costs. The "flex" capacity pool is small. New incremental compute transacts at current elevated prices. Cloud and chip vendors (not model labs) capture most of the margin uplift.

• [00:34:00] The bottleneck shifts back to chips — As power and data center constraints ease (both shorter lead-time supply chains), the binding constraint through 2030 becomes the semiconductor supply chain itself. TSMC fab construction: 2-3 years. Amazon built a data center in 8 months. The mobile/PC industry has already shifted as much capacity toward AI as it can; there is no remaining slack to redeploy.

• [00:37:00] ASML EUV: the actual ceiling — 1 gigawatt of Rubin-generation compute needs ~2 million EUV wafer passes, which works out to 3.5 EUV tools. ASML produces ~70 tools/year now, scaling to ~80-100 by 2030. Cumulative installed base of ~700 tools by 2030 implies a theoretical ceiling of ~200 GW if all capacity were AI-allocated. Sam Altman's 50 GW/year target represents roughly 25% market share — within reach but implies every other use case gets squeezed.

• [01:00:00] Inference performance gap is 20x, not 3x — On identical DeepSeek / Kimi K2.5 workloads, Blackwell outperforms Hopper by roughly 20x, not the 2-3x raw flops delta implies. The gap comes from inter-chip networking bandwidth, architectural improvements in how compute is organized within the scale-up domain, and memory bandwidth — all non-portable to older nodes even with redesigned chips.

• [01:16:00] Memory crunch cascades to consumer electronics — HBM uses 3-4x more wafer area per bit than DRAM; the same EUV shortage that gates AI compute is now redirecting memory supply away from consumer devices. iPhone memory cost up ~$150/unit; low-end smartphone volumes projected to fall from 1.4B to 500-600M units/year. ~30% of hyperscaler 2026 capex goes to memory. Memory fab response is 2+ years behind demand inflection because vendors stopped building during the 2023 low-price period.

• [01:33:00] Power is solvable — not the bottleneck — 16+ manufacturers cover power generation from gas alone (combine cycle, aeroderivatives, ship engines, reciprocating engines), plus Bloom Energy fuel cells and solar+battery. Patel estimates hundreds of GW could come from behind-the-meter sources by end of decade. Power supply chain is "way more simple than chips." Cost doubling on power barely moves the needle on per-token economics since power is a small fraction of TCO.

• [01:54:00] Space data centers: wrong answer to the wrong problem — Power is free in space, but power isn't the constraint. Chips are. Space deployment adds 6+ months of latency to GPU deployment (deconstruction, shipping, reassembly, testing) — 10% of a 5-year GPU useful life. Inter-satellite bandwidth (100 gbps Starlink links) is orders of magnitude below InfiniBand (400 gbps per GPU × 72 GPUs per rack). Space makes sense post-2030 when chips are no longer binding, not now.

• [02:10:00] Small models compound faster — The feedback loop at AI labs favors smaller models: faster RL rollouts, faster research iteration, quicker deployment into the next-model build cycle. Google is the exception — homogeneous TPU fleet enables 4,000-9,000-chip scale-up domains that make large-model RL tractable. For heterogeneous fleets (Anthropic, OpenAI), smaller models that can be RL'd rapidly win.

• [02:14:00] Information asymmetry and who profits — SemiAnalysis sells to ~60% industry clients and ~40% hedge funds. Leopold Aschenbrenner is described as "the only client who tells me our numbers are too low." The memory crunch trade was visible a year before prices moved if you believed the KV-cache-grows-with-context-length logic and tracked fab construction data. The market only prices these constraints once they're visible to generalists.

• [02:28:00] Taiwan risk is existential for compute scaling — Airlifting engineers wouldn't save the world semiconductor position; TSMC tools themselves use chips made in Taiwan (circular dependency), and rebuilding equivalent fab capacity elsewhere would take years and cost hundreds of billions. A Taiwan disruption drops incremental AI compute from hundreds of GW/year to roughly 10-20 GW across Intel and Samsung.

Notable claims

• [00:02:00] ~20 GW of incremental critical-IT AI compute added in the US this year; big-four hyperscaler capex ~$600B with additional supply-chain spending getting total toward ~$1T.

• [00:40:00] $1.2B in EUV tooling underpins $50B+ in data center capex per gigawatt of AI compute — the most constrained resource in the chain is also among the cheapest as a fraction of system cost.

• [01:00:00] Hopper vs. Blackwell inference performance on DeepSeek/Kimi: 20x difference despite both being on the same process node — driven by networking topology, scale-up domain size, and memory bandwidth, not flops.

• [01:21:00] HBM4 bandwidth per stack: ~2.5 TB/s. DDR5 bandwidth in equivalent shoreline area: ~64-128 GB/s. Roughly 20x gap per chip edge — explains why DDR can't substitute for HBM in high-throughput inference.

• [01:23:00] ~30% of hyperscaler capex in 2026 is going to memory alone.

• [01:24:00] If DRAM prices triple: ~$150 added cost to an iPhone BOM. Apple will likely pass most of it to consumers. Low-end Android OEMs (OPPO, Xiaomi) are reportedly cutting mid/low-range volumes by half because their margins are too thin to absorb it.

• [01:43:00] Theoretical 2030 ceiling: ~200 GW of AI compute if all ASML output is AI-allocated. Current global critical-IT capacity: ~20 GW. Sam Altman's target of 52 GW/year by 2030 requires ~25% of all semiconductor fab output.

• [02:22:00] If Huawei had not been banned from TSMC in 2019, Patel assesses they would likely be TSMC's largest customer today and potentially build a better accelerator than Rubin at 3nm — they have the software, networking expertise, AI talent, end market, and their own fab infrastructure.

• [02:29:00] Post-Taiwan-disruption scenario: global incremental AI compute capacity drops to ~10-20 GW/year (Intel + Samsung) vs. hundreds of GW on the current trajectory.

Guests

Dylan Patel — Founder and CEO of SemiAnalysis, a semiconductor and AI infrastructure research firm. Patel tracks global wafer orders, data center construction, fab capacity, and tool supply chains at a level of granularity that most Wall Street and industry participants lack. SemiAnalysis sells research and data to AI labs, hyperscalers, semiconductor companies, and hedge funds. Patel is regularly cited by investors and operators as the primary source for supply-chain signal on AI compute.

Mapping against Ray Data Co

Capital cycle thesis — direct validation. RDCO's investment focus is the chip-fab/memory capital cycle. Patel's framework maps precisely: ASML is the lynchpin (monopoly, artisanal supply chain, not raising prices despite leverage), memory vendors are the secondary chokepoint (underbuilt for 3+ years, now tripling prices), and TSMC is the orchestration layer. The "who holds the cards" analysis — Nvidia > cloud > memory vendors > TSMC/ASML — is the right mental model for positioning in that cycle.

Memory trade signal. The KV-cache-grows-with-context argument that justified a memory long is still live: reasoning models, long-context workloads, and agentic deployments all expand KV cache demand. Memory fab capacity doesn't respond until late 2027-28. Any enterprise client building long-horizon AI deployment plans should be told: the memory crunch is structural, not cyclical.

Client advisory — compute access strategy. The 5-year early-commitment margin advantage is directly applicable to enterprise clients evaluating cloud GPU contracts. The message: committing to compute at today's prices, even if it feels aggressive, is likely the correct long-term economic posture — same logic OpenAI used vs. Anthropic's conservative approach.

Inference tier selection. The 20x Hopper-to-Blackwell gap changes the cost-benefit math for enterprise AI deployments. Clients on H100/H200 clusters should not expect that "newer models will just be cheaper to run" — the gap between Hopper and Blackwell inference economics is architectural, not just a process-node improvement.

AI advisory credibility. Patel's bottleneck progression framework (2022: CoWoS → 2023: power/data centers → 2025-26: clean rooms → 2028+: EUV/ASML tools) is the most defensible public analysis of AI infrastructure constraints. RDCO can adopt this as a shared vocabulary with technically sophisticated clients without needing to cite a source they don't recognize.

Geopolitical framing. "Fast timelines = US wins; slow timelines = China wins" is a clean client-facing heuristic for why AI urgency matters beyond pure economics. Patel's Taiwan risk section is especially useful: loss of Taiwan doesn't just create supply disruptions — it could cut incremental AI compute capacity by 90%+ for years.

Sponsorship

Three ad reads in this episode:

• Mercury (~[00:17:00]) — banking and 1099 platform for startups; Dwarkesh reads the ad.
• Labelbox (~[00:54:00]) — voice model evaluation pipeline; positioned at AI teams running RLHF/voice workloads.
• Jane Street (~[01:31:00]) — hiring ad targeting ML researchers and quantitative traders.

No equity or advisory relationships declared between Patel/Dwarkesh and any sponsor. No plugs for SemiAnalysis paying clients during the interview.

Related

• semiconductor-capital-cycle
• ai-compute-economics
• [[2026-03-13-dwarkesh-dylan-patel-ai-compute-bottlenecks-transcript]]
• [[investing-markov-capital-cycle]]