"Dwarkesh Goes Inside Jane Street's Latest AI Data Center" — Jane Street

Why this is in the vault

Filed as a primary-source eyewitness anchor for the [[2026-05-17-power-cycle-v1]] and [[2026-05-17-memory-cycle-v1]] theses. The Jane Street physical-engineering team gives a quant-shop's-eye view of what GB300 NVL72 deployment actually looks like in the field: 140 kW/cabinet (vs 10-40 kW air-cooled baseline = ~4-14x density jump), liquid-cooling retrofit of a facility never designed for it, ultrasonic flow meters cabinet-by-cabinet, 8,000 km of fiber, and — most load-bearing — explicit confirmation that the binding constraint is power allocated from the utility, not compute, not money. Ron Minsky says it directly: "you have some power allocated from the utility ... you still have to respect that overall value but you want to ride as close to it as possible." That sentence is the [[2026-05-12-innermost-loop-ai-infrastructure]] power-bottleneck thesis stated by an operator who is currently retrofitting a live training cluster to deal with it. Not a slide; a tour.

Also filed for the operator-craft signal: Jane Street is a market-maker, not a hyperscaler — yet they are now running a 4,032-GPU GB300 cluster on liquid cooling in Texas. The "everyone is a hyperscaler now" pattern shows up in capex flow data; this is the qualitative analog. Useful as Sanity Check anchor reference (not a topic) and as ground-truth corroboration for any future deep-research brief on power/memory/cooling supply chains.

Episode summary

A 15-minute walking tour of Jane Street's Texas AI training facility, hosted by Dwarkesh Patel and led by Ron Minsky (co-head of technology) and Daniela Corvo (head of physical engineering). The facility is a retrofit — built earlier for traditional air-cooled racks, now converted to support 200 kW liquid-cooled GB300 NVL72 cabinets. The conversation walks through cooling fluid loops (18°C chiller water, propylene glycol mix, 25-micron filtration, ultrasonic flow meters, leak-detection ropes), power distribution (busways, breaker panels, oversubscription philosophy, hardware-enforced load shedding), networking density (copper for the fastest hops, fiber for everything else, 8,000 km total), and the operator's mental model: opportunity cost of compute dominates hardware cost; physical infrastructure (transformers, chillers) is growing faster than the compute footprint itself.

Closes with a "20 years ago" reminisce — the original Jane Street compute cluster was six Dell boxes called "the Hive" stacked on the office floor. Promises a longer Dwarkesh podcast follow-up on Jane Street's low-latency trading systems (sub-100ns packet turnaround).

Key arguments / segments

• [00:00:00] Cluster identification: GB300 NVL72, training cluster. Jane Street confirms LLM training plus "custom architectures more adapted to the trading problems and trading data sets." A trading firm running a frontier-scale training cluster is itself notable.
• [00:00:45] Retrofit shock: 140 kW/cabinet vs 10-40 kW air-cooled baseline. GB300 cabinets consume at peak ~140 kW. ~85-90% of heat removed via cold plates; ~15% of cabinets retain air cooling for legacy/hybrid load. Sleds slide in and auto-connect to liquid supply, liquid return, and 54V power simultaneously.
• [00:01:30] Liquid cooling risk surface. Leak-detection ropes inside servers + under-floor sensors with isolation valves. Acknowledgment: "if something here fails, you are at risk of destroying the server. How often is there a leak? Not often, but this stuff is new." Forward-looking risk, not solved.
• [00:03:00] Optionality as design principle. Minsky frames the retrofit as deliberate: "this place was built in a kind of intermediate point when we knew we had to scale up a lot but we didn't know what the shape of the coming compute was. ... importance of optionality." Useful counter to the "no one saw this coming" narrative — Jane Street did, and built for ambiguity.
• [00:03:45] Fungible fluid topology. 18°C chiller water from roof; same fluid serves both liquid-cooled and air-cooled paths inside the data hall. Per-cabinet flow balancing via ultrasonic flow meters and metering valves so end-of-row cabinets don't starve.
• [00:04:20] Technical water loop = 25-micron filtered, 25% propylene glycol. Filtration prevents cold-plate clogging; glycol inhibits bacterial/algae growth that would also clog cold plates. Bacteria is now a server reliability concern.
• [00:05:30] Cluster scale: 4,032 GPUs in 56 racks. Single facility, single training cluster. For reference: a Hopper-era H100 cluster of comparable GPU count would have occupied substantially more floor space at lower kW/rack.
• [00:06:15] Opportunity cost dominates hardware cost. Minsky: "compute is relatively inelastic ... people are all kind of bidding for the same compute ... it just becomes incredibly expensive because the stuff that we get out of this is super valuable to the business. And so the opportunity cost tends to dominate the hardware cost." Internal Jane Street pricing model.
• [00:07:00] Power as the binding constraint — load-bearing quote. "If this facility is connected to the grid and you presumably had asked the grid beforehand for a certain amount of power and now you move to much denser compute. How are you able to get the power in here?" Answer: shrink the compute footprint inside the existing power envelope. "you still have to respect that overall value but you want to ride as close to it as possible." This is the power-cycle thesis stated by an operator.
• [00:08:00] Latency physics: copper beats fiber. "Light moves more slowly in fiber than electrons move in copper." Fastest internal links are copper; fiber for everything else. 8,000 km of fiber total in this single deployment.
• [00:09:00] Pre-build for headroom + oversubscription discipline. Power distribution is intentionally overbuilt for re-allocation. Cooling is fungible; power is not (breakers, current limits). Oversubscription is the philosophy but tripping a breaker mid-training = lost run progress.
• [00:10:30] NVIDIA's LPS (Load Power Shaping) systems. New cabinet generation includes bulk capacitance + software to flatten peak-vs-average load profile. Hardware-enforced safety net beneath the software oversubscription. Topology-aware monitoring can shut down nodes pre-emptively to avoid breaker trip.
• [00:12:00] Buffer tanks = thermal battery. Liquid cooling needs survival window if chillers restart. Buffer tanks store cooling capacity to bridge chiller-restart window without GPU thermal kill.
• [00:13:30] Density inversion: compute footprint shrinks, supporting infrastructure grows. "Where you have the compute is getting smaller and smaller and the places where you're supporting that compute is getting larger and larger. The infrastructure, the transformers, the chillers are getting bigger and bigger." This is the physical instantiation of the innermost-loop thesis — the data-center-infra layer is structurally outgrowing the chip layer.
• [00:14:00] The Hive: six Dell boxes on the office floor (20 years back). Origin story for Jane Street compute. Includes the great anecdote about a cleaner accidentally unplugging the trading system mid-day with a vacuum.
• [00:15:30] Tease for podcast follow-up. Sub-100ns packet turnaround in their fastest HFT systems — Dwarkesh promises to dig in on the podcast.

Notable claims

1. GB300 NVL72 cabinet peak power: ~140 kW — vs 10-40 kW for traditional air-cooled. 4-14x density jump. (00:00:45)
2. 85-90% of heat removed via cold-plate liquid cooling, ~10-15% still air-cooled inside the same sled. (00:01:00)
3. Chiller water supply: ~18°C, fungible between liquid-cooling loop and air-cooling air-handler loop. (00:03:45)
4. Technical water loop: 25-micron filtration + 25% propylene glycol for bacterial inhibition. Bacterial growth in cooling water now a real reliability concern. (00:04:30)
5. Single facility: 4,032 GPUs in 56 racks, ~8,000 km of fiber. (00:05:30, 00:08:30)
6. Copper > fiber for the fastest network hops (electrons-in-copper travel faster than light-in-fiber). (00:08:00)
7. Opportunity cost of compute > hardware cost at frontier-scale internal allocation. Jane Street-internal compute "real crunch" reported by Minsky. (00:06:15)
8. Power from utility is the binding constraint; facility design accommodates by shrinking compute footprint inside fixed power envelope. (00:07:00)
9. NVIDIA LPS (Load Power Shaping) = bulk capacitance + software peak/average flattening rolling out in new cabinet generation to keep load profile flat. (00:10:30)
10. Cooling is fungible (oversized pipes); power is not (hard breaker limits). Different engineering tolerances for the two utilities. (00:09:30)
11. Buffer-tank thermal-battery design bridges chiller-restart window. Operational pattern, not yet widely discussed in public infra writeups. (00:12:00)
12. Density inversion law: compute footprint is shrinking; support-infrastructure footprint (transformers, chillers, breaker panels) is growing. (00:13:30)
13. Jane Street's fastest HFT systems target sub-100ns packet turnaround. Tease — to be elaborated in Dwarkesh podcast follow-up. (00:15:30)

Guests

Ron Minsky — co-head of Technology at Jane Street. Long-tenured Jane Street technologist; OCaml community presence; the strategic-framing voice in this conversation. Tracked-author candidate: MEDIUM. Worth adding to Contact Candidates DB as a "operator-side voice on quant-shop AI infrastructure" — there are very few public-facing speakers from buyside quant tech orgs at this depth. The Dwarkesh podcast follow-up will be the better trigger; if that conversation is substantive, promote.

Daniela Corvo — head of physical engineering at Jane Street. The cooling/power/leak-detection details came from her. First public Jane Street appearance I can find; strong primary-source authority on data-center physical retrofit. Tracked-author candidate: WATCH-ONLY — wait for a second appearance before promoting; one-off may not generalize.

Dwarkesh Patel — host, already vault-tracked. Filming on-location at a trading firm's data center is a notable Dwarkesh format extension (was previously studio-bound; the April 2026 Reiner Pope blackboard format was the prior format shift). Pattern: Dwarkesh is increasingly getting access to operator-grade environments.

Mapping against Ray Data Co

Strength: STRONG. Reinforces three active investing theses directly, with eyewitness operator language:

• [[2026-05-17-power-cycle-v1]] — this video provides the operator-side qualitative anchor for the power-cycle thesis. Minsky's quote at [00:07:00] ("you have some power allocated from the utility ... ride as close to it as possible") is the buy-side version of what Talen/Constellation/Vistra are signing 17-year PPAs to satisfy. The thesis was sourced from supply-side data (queue wait times, capex aggregates, PPA cadence). This video confirms it from the demand side: even a quant-shop on the smaller end of GPU buyers is now power-constrained, not capital-constrained. Cross-link as anchor evidence.
• [[2026-05-17-memory-cycle-v1]] — GB300 NVL72 is the cabinet whose HBM density drove the density inversion. The "4-14x kW jump per cabinet" is the physical instantiation of "every HBM wafer crowds out 3 commodity DRAM wafers" — Jane Street had to liquid-cool a building to deploy them. Reinforces the HBM-supply-as-binding-constraint claim with operator language.
• [[2026-05-12-innermost-loop-ai-infrastructure]] — the [00:13:30] density inversion quote ("compute footprint shrinks, supporting infrastructure grows") is the physical instantiation of the innermost-loop framework. Layer 3 (data center infrastructure: Vertiv, GE Vernova, Bloom, Oklo, Fluence) literally larger in floor area than Layer 2 (chips). This is the visual evidence behind the thesis's structural claim that infrastructure has become the load-bearing layer.

Cross-link with prior dwarkesh + AI-infra reference set: pairs especially well with [[2026-04-29-dwarkesh-reiner-pope-gpt5-claude-gemini-training]] — Reiner derived from first principles why memory-bandwidth (not compute) is the inference bottleneck; this video shows the physical infrastructure side of the same constraint. The two together form a strong demand-supply-physical-constraint triad. Also pairs with [[2026-04-15-dwarkesh-jensen-huang-nvidia-moat]] for the NVL72 design context.

Not Sanity Check topic material per [[feedback_no_derivative_sanity_check_pieces]] — this is reference/anchor evidence, not a re-frame. Don't pitch a piece. Cite it inside a future re-frame on "the bottleneck has moved to power" or "everyone is a hyperscaler now."

Related

• [[2026-05-17-power-cycle-v1]]
• [[2026-05-17-memory-cycle-v1]]
• [[2026-05-12-innermost-loop-ai-infrastructure]]
• [[2026-04-29-dwarkesh-reiner-pope-gpt5-claude-gemini-training]]
• [[2026-04-15-dwarkesh-jensen-huang-nvidia-moat]]
• [[2026-04-19-dwarkesh-satya-nadella-microsoft-agi]]