"Terence Tao – How the world's top mathematician uses AI" — Dwarkesh Patel

Why this is in the vault

The highest-rated working mathematician alive gives a granular, unsentimental account of where AI actually helps and fails in frontier knowledge work — making this primary evidence for how elite practitioners are integrating AI tools, and where the real capability gaps remain.

Episode summary

Terence Tao and Dwarkesh Patel trace AI's role in mathematics from first principles, using Kepler's discovery of planetary motion as a frame for how hypothesis generation, data, and verification each contribute to scientific progress. Tao describes his hands-on experience using AI to accelerate auxiliary tasks while keeping pen-and-paper for the core hard problems, and explains why the AI bottleneck is no longer idea generation but cumulative reasoning and verification at scale. The conversation closes with Tao's perspective on career advice for mathematicians and a measured but genuinely uncertain outlook on the decade ahead.

Key arguments / segments

• [00:04:00] Dwarkesh frames Kepler as "a high-temperature LLM" — cycling through random hypotheses against Brahe's data until one stuck — to open the question of whether AI can play a similar empirical discovery role.
• [00:12:00] Tao argues AI has driven the cost of idea generation "down to almost zero," analogous to how the internet drove communication costs to zero — remarkable, but it shifts the bottleneck to verification and validation, not supply of ideas.
• [00:30:00] On Erdős problems: roughly 50 of 1,100 have been solved with AI assistance, producing an initial burst of wins that has since plateaued; pure one-shot AI solutions have largely stopped materializing.
• [00:35:00] AI excels at breadth — applying every known technique across many problems simultaneously — while humans excel at depth. Tao argues science needs to be redesigned around this complementarity rather than treating AI as a human replacement.
• [00:43:00] The ~50 Erdős problems AI solved were predominantly ones with almost no prior literature; Tao estimates AI has roughly a 1-2% success rate per individual problem, but scale makes the wins visible and the failures invisible.
• [00:47:00] Tao confirms his 2023 prediction that AI would be a "trustworthy co-author if used correctly" by 2026 looks accurate; he reports being 5x faster on auxiliary tasks (plots, code, literature search, LaTeX formatting) but unchanged on the core hard problems.
• [00:49:00] He distinguishes "artificial cleverness" from "artificial intelligence": current models lack the cumulative, session-spanning buildup of partial progress that characterizes real mathematical collaboration — each session starts from zero.
• [00:57:00] Lean proofs can be atomically decomposed and studied lemma-by-lemma; Tao is less worried about incomprehensible AI-generated proofs than others, because post-processing and refactoring tools already exist once the artifact is produced.
• [01:00:00] He floats the idea of a formal or semi-formal language for mathematical strategies (not just proofs), analogous to how Lean formalized deductive proof — currently a wish rather than a plan, but motivated by how much Lean has already improved AI training for proofs.
• [01:21:00] Career advice for early mathematicians: embrace adaptability, expect that traditional boundaries will dissolve, and stay open to non-traditional pathways — AI tools may allow high-school-level contributors to touch frontier math in ways not previously possible.

Notable claims

• [00:12:00] "AI has basically driven the cost of idea generation down to almost zero" — Tao's framing; the bottleneck is now sorting and validating ideas, not generating them.
• [00:44:00] AI success rate on Erdős problems is approximately 1-2% per problem; aggregate wins look impressive only because they are applied at scale and only winners get broadcast.
• [00:46:00] Tao's 2023 prediction — AI would be a trustworthy co-author in mathematics by 2026 "if used correctly" — held up to his own satisfaction by the time of this conversation.
• [00:48:00] Specific tasks like generating plots, running deeper literature searches, and reformatting LaTeX now take 5x less time; but the core mathematical insight work is "about the same" and still done with pen and paper.
• [01:18:00] Within a decade, AI will likely handle much of what mathematicians currently spend time on — but hybrid human-plus-AI teams will dominate frontier work longer than fully autonomous AI; full replacement is stochastic and requires additional breakthroughs.
• [01:06:00] If the Riemann hypothesis proved false, Tao would expect cryptography based on prime numbers to be rapidly abandoned — because a hidden pattern in the primes would imply more unknown patterns that could be exploited.

Guests

Terence Tao — Distinguished Professor of Mathematics at UCLA and Fields Medal recipient (2006); widely regarded as the most accomplished active research mathematician in the world. He works across an unusually broad range of subfields including harmonic analysis, partial differential equations, combinatorics, analytic number theory, and compressed sensing. He maintains an influential mathematics blog and has been an early and serious public commentator on AI-assisted mathematics.

Mapping against Ray Data Co

Several threads connect directly to RDCO's operating context:

Idea generation is no longer the constraint. Tao's framing that AI has commoditized hypothesis generation maps exactly onto what RDCO sees in client analytics and data work: the tools to generate candidate insights are cheap; the bottleneck is rigorous validation and turning raw signals into decisions stakeholders trust. This is a strong narrative anchor for RDCO's positioning — "we don't sell you more ideas, we help you verify and act on them."

Breadth vs. depth as a design choice. Tao explicitly calls out that current AI excels at breadth (applying all known techniques to all problems simultaneously) while humans excel at depth. RDCO operates in exactly this mode on client engagements — AI handles broad pattern scanning, humans supply the domain judgment and accumulated context that a model loses between sessions. This is worth articulating explicitly in RDCO positioning materials.

The 1-2% success rate framing is a useful client-facing heuristic. When Tao describes AI solving Erdős problems at 1-2% success per problem but looking extraordinary in aggregate, this captures something important about production AI deployments: you need volume and filtering infrastructure, not just a capable model. RDCO's value proposition in data stack work is building exactly that filtering and reliability layer.

Elite practitioners use AI for auxiliary tasks, not core judgment. Tao's description of AI handling plots, literature search, and LaTeX reformatting while he keeps pen-and-paper for the hard parts mirrors what high-performing knowledge workers across domains are reporting. This pattern — AI as a junior assistant that handles surface tasks, human as the irreplaceable judgment layer — is a useful frame for scoping AI integration projects with clients.

Cumulative reasoning gap is still real. The observation that models start from zero each session and cannot build up shared partial progress across a collaboration is a genuine constraint that affects any RDCO engagement involving extended, multi-session AI-assisted analysis work. Understanding this limitation helps avoid over-promising on autonomous AI workflows.

Related

• ai-capability-research-frontier-math
• knowledge-work-ai-integration-patterns
• llm-limitations-reasoning-and-verification
• claude-usage-sops