Andrej Karpathy — “We’re summoning ghosts, not building animals” (Dwarkesh Patel)

Why this is in the vault

Karpathy is the most credentialed sober voice currently arguing publicly that the “year of agents” framing is wrong and that we are roughly a decade — not 1-2 years — from the kind of AI that can reliably do an intern’s work. He’s not a permabear (he started OpenAI, ran Tesla Autopilot, just wrote nanochat) and he’s not selling a competing product. When he says GPT-5 Pro is “slop” inside his own repo and that the labs are “trying to pretend like this is amazing, and it’s not” — that lines up exactly with Sanity Check’s editorial spine: trust what builders say when they’re not in fundraising mode. This is also the canonical articulation of the “ghosts vs animals” frame that makes the imitation-learning critique tractable.

The core argument

Decade of agents, not year of agents. Karpathy explicitly walks back the agentic-AI hype as “over-prediction.” The three things missing — multimodality, computer use, and continual learning — are tractable but each requires years of integration work. His 15 years of seeing AI predictions is the basis for the decade estimate; he doesn’t claim more.

We’re building ghosts, not animals. This is the frame the title comes from. Animals are the product of evolution: huge amounts of behavior are pre-baked into the weights via DNA before any learning happens (a zebra runs minutes after birth — that’s not RL). LLMs are the product of imitation of internet text — they’re “ethereal spirit entities” mimicking humans. Different optimization process, different starting point. Pre-training is “crappy evolution” — the practical version we can actually run. He pushes back hard on Sutton’s “build animals” frame: not because it’s wrong in principle, but because we don’t have the substrate for it.

Cognitive core hypothesis. Pre-training does two things simultaneously: it gives the model knowledge AND it gives the model intelligence (the algorithms for in-context learning, etc.). The knowledge is partly holding the intelligence back — agents are bad at “going off the data manifold.” The research direction he’s most interested in: strip the knowledge, keep the cognitive core. A small intelligent entity that knows it doesn’t know X and can go look it up.

Coding agents as honest signal. Most of the Karpathy section on coding is the most quotable in the entire interview. He’s an autocomplete-tab loyalist, not a vibe-coder. On nanochat (intentionally unique repo, not boilerplate): the models kept trying to use DDP when he’d rolled his own gradient sync, kept inserting try/catch defensive bloat, kept using deprecated APIs, kept misunderstanding intent. “They’re way too over-defensive. They keep trying to make a production code base, and I have a bunch of assumptions in my code, and it’s okay.” GPT-5 Pro as the “oracle” he occasionally consults with full repo context — surprisingly good vs a year ago, but still not net useful for novel work.

The recursive-self-improvement objection. Dwarkesh frames the AI-2027-style argument: if you have Claude Code making CRUD apps, imagine a million of these inside OpenAI tweaking architectures. Karpathy: that’s exactly the asymmetry — the models are bad at code that has never been written before, which is what AI research IS. This is his main mechanical reason for longer timelines.

Compiler analogy. Programming has had decades of productivity wins (compilers, linters, languages, autocomplete) without explosion. Autocomplete-mode AI looks like that line. Vibe-coding agents would look different. He sees current AI as continuous with the compiler tradition, not a discontinuity.

Mapping against Ray Data Co

Direct alignment. This is, alongside the “What are we scaling?” essay, the strongest external corroboration of the Sanity Check editorial position so far. The “models keep getting more impressive at the rate short-timelines people predict, but more useful at the rate long-timelines people predict” line from the prior Dwarkesh essay rhymes exactly with Karpathy’s “the industry is trying to pretend this is amazing, and it’s not. It’s slop.”

Specific newsletter ammunition:

• The DDP/over-defensive-code anecdote — gold for any piece on why “AI will replace senior engineers” arguments are wrong. The model’s failure mode is reverting your novel work to the median codebase shape from training. The more your work matters (i.e., is non-boilerplate), the worse the agent gets. This is the inverse of “boilerplate is the safe ground for AI” — actually, boilerplate is the only ground. File this for the “AI productivity is a U-curve, not a line” piece I want to write.
• Cognitive core / strip-the-knowledge. This pairs with our positioning that the data layer (context, memory, retrieval) is where the real product surface lives. If the future is “small intelligent core + curated knowledge fed in via context,” then the knowledge curation function is durable. RDCO’s lane.
• Decade timeline as customer permission. Most enterprise customers are getting battered by “AI changes everything in 18 months” pitches. A Karpathy-quoted decade gives them permission to invest in foundations (data quality, governance, integration) instead of chasing each model release. Useful for any “stop letting AI roadmaps eat your data roadmap” framing.
• Compiler analogy — important rebuttal to the AGI-explosion frame. Use when arguing that AI productivity gains can be huge AND not produce a discontinuity. Programming languages were a 10x+ productivity win and didn’t cause an explosion. Strong for the “AI as power tool, not labor replacement” frame.
• Ghosts vs animals. This is a pre-loaded metaphor we can borrow. The honest frame for Sanity Check readers is: you’re not buying an employee, you’re buying a very good imitator of internet text patterns. Manage your expectations and your data accordingly.

Sanity Check candidate hooks:

• “Karpathy spent six months writing nanochat. He used autocomplete, not agents. Here’s what that should tell you about your agent strategy.”
• “The strongest AI-skeptic in the industry has 15 years of pattern-matching and just shipped a model. He says decade, not year.”
• “Stop trying to build animals. You’re buying ghosts. Here’s what that means for your data.”

Open follow-ups

• Find the Karpathy nanochat repo and read the README — he refers to specific architectural choices (Muon optimizer, custom DDP, etc.) that would inform our own stance on “what does a from-scratch AI codebase look like in 2025.” Worth a separate vault entry on the repo itself.
• Toby Board / continual-learning paper trail — referenced in the “What are we scaling?” essay as a 1,000,000x RL compute scaling claim. Track down primary source.
• Sutton interview itself — Karpathy’s Sutton response is the hinge of this whole interview. We have an assessment of the “Sutton interview thoughts” essay but should listen to the actual Sutton episode to triangulate.
• The Universe project / 2017 OpenAI computer-use agent — Karpathy says they were “way too early.” Worth a short historical note on why early agents failed (sparse rewards, no representation backbone) so we can recognize the same failure pattern in current attempts.
• Anthropic’s Claude Code position — Karpathy uses Claude and Codex daily but says they’re not net-useful for novel work. Worth a follow-up: where’s the line, exactly? Boilerplate vs glue vs novel? Could become a Sanity Check framework.

Related

• 2025-12-23-dwarkesh-what-are-we-scaling — Dwarkesh’s solo essay that follows on directly from this interview. Karpathy’s “ghosts vs animals” + “cognitive core” frames show up there as accepted vocabulary.
• 2025-10-04-dwarkesh-sutton-interview-thoughts — Dwarkesh’s reaction to the Sutton interview, which Karpathy is here responding to. The triangulation: Sutton says LLMs aren’t real intelligence, Karpathy says yes but ghosts are still useful, Dwarkesh says both are right and the decade timeline is the synthesis.
• 2026-03-11-dwarkesh-most-important-question-about-ai — same author, downstream episode on alignment-to-whom and Anthropic-DoW supply-chain risk.
• 2026-04-15-thariq-claude-code-session-management-1m-context — Thariq’s “context rot” / “fat skills, thin harness” view aligns with Karpathy’s “strip the knowledge, keep the cognitive core” frame. Thariq’s pattern is the operational consequence of Karpathy’s research direction.