IndyDevDan — Agent Experts: Finally, Agents That ACTUALLY Learn
Why this is in the vault
19-minute lesson where Dan articulates the agent expert pattern — an agent paired with a YAML “expertise file” that the agent reads BEFORE doing work and writes back AFTER doing work, so the agent’s mental model evolves in place. The vault keeps it because it makes one specific architectural claim that nothing else in the IndyDevDan corpus does: the expertise file is NOT a source of truth — it is a working memory file, distinct from code, comments, PRDs, and plans. This is the missing operational answer to the “agents forget” problem that everyone (Anthropic, every harness vendor, the founder) has been gesturing at but not solving cleanly. Pairs directly with the 2026-04-19-indydevdan-cracked-claude-agent-skills piece — skills are the static building blocks; agent experts are skills with a self-improvement loop bolted on. Also closes a loop opened by 2026-04-15-thariq-claude-code-session-management-1m-context — Thariq said context windows degrade with size; the expertise-file pattern is one concrete answer for how a long-running agent system avoids needing a giant context window in the first place.
Core argument
- Memory files, prime prompts, sub-agents, and skills all share one defect: they must be MANUALLY updated. This is the structural reason agents stop improving past a certain point — the human is the bottleneck on the agent’s growth. Every other mechanism Dan lists is steerable but not self-steering.
- An agent expert is a self-improving template metaprompt. Three-step lifecycle: (a) read expertise file, (b) execute with mental model + validate against the codebase as ground truth, (c) self-improve — append/update the expertise file based on what was learned during the run.
- The expertise file is a working mental model, NOT a source of truth. Critical distinction. The CODE is the source of truth. The expertise file is a YAML data structure that captures “what this agent has learned about a specific area of the codebase” — analogous to a senior engineer’s intuition about a system, not the system’s documentation.
- Meta-agentics — meta-prompt, meta-agent, meta-skill — is the foundation layer. The system that builds the system. Dan demos a meta-prompt creating a question-with-mermaid-diagrams prompt, a meta-agent creating a planner agent, a meta-skill creating a start-orchestrator skill. Every codebase Dan creates has these. Three times marks a pattern; pattern marks a meta-tool.
- Expertise files give just-in-time context, not always-on context. Memory files (CLAUDE.md, AGENTS.md) are GLOBAL FORCED CONTEXT — they always load. Expertise files are LAZY — only the agent that needs the websocket expertise reads the websocket expertise file. This is how a 10-expert codebase doesn’t blow every agent’s context window with 9 files of irrelevant knowledge.
- Scale per-expert with parallel agent fan-out. Three websocket experts answering the same question gives higher confidence in the answer than one expert answering once. Compute scaling at the expert level, not just the agent level.
- The Core Four (context, model, prompt, tools) still applies — agent experts are just the prompt+context layer specialized. Dan keeps returning to the Core Four as the only abstraction that matters. Skills, custom agents, MCP servers — all decompose to manipulations of these four.
- Self-improve must be the THIRD step of the three-step workflow. Plan, build, self-improve. If self-improve is optional or skipped, the agent never learns. The discipline of always running self-improve at the end is what turns a generic agent into an expert.
Mapping against Ray Data Co
- Expertise files are the structural answer for RDCO’s vault skill ecology. Right now, ~/.claude/skills/ contains 50+ skills, each of which uses the same global CLAUDE.md and SOUL.md context regardless of relevance. The expertise-file pattern says: each skill should also have a
~/.claude/skills/<skill>/expertise.ymlthat records what THAT skill has learned (e.g., /process-newsletter knows what subject lines from each sender historically meant low-signal vs high-signal; /check-board knows which Notion property quirks have bitten us). Worth a 2-week experiment on one skill (suggest /process-newsletter) before generalizing. - Self-improve maps directly to the post-cycle log step in /check-board. Every cycle currently writes a log entry; the expertise-file pattern says each cycle should ALSO update the skill’s expertise file with one or two learned facts. Concrete: if a backfill cycle hit a yt-dlp failure mode, that failure mode + workaround should be appended to /process-youtube/expertise.yml so the NEXT cycle reads it before starting.
- The “expertise file is NOT a source of truth” rule is the operational guardrail SOUL.md needs. SOUL.md is currently silent on the difference between the founder’s stated decisions (source of truth — preserved verbatim) and the agent’s learned patterns (mental model — overwritten as understanding deepens). The vault has been mixing these. Worth adding a one-paragraph distinction to SOUL.md: founder utterances = immutable; agent learned patterns = mutable.
- Meta-agentics validates the existing “skill that creates a skill” practice. RDCO already has /improve (improves an existing skill) and skill-creator. These are meta-agents in Dan’s vocabulary. The missing piece is the meta-PROMPT — a prompt that generates new prompts for one-off tasks. Useful for /research-brief variations or /process-newsletter tuning. ~1 hour to scaffold.
- Just-in-time context > always-on context is a direct critique of CLAUDE.md sprawl. CLAUDE.md is currently 200+ lines and loads on EVERY agent invocation, including ones that don’t need 90% of it (e.g., a Vercel deploy doesn’t need the iMessage reply rules). Expertise-file pattern says: split CLAUDE.md into a tiny core + per-skill expertise files that load only when the relevant skill runs. Significant context savings across long sessions.
- Three-times-marks-a-pattern is the right rule for promoting one-off scripts to skills. RDCO has been creating skills somewhat eagerly (the discover-sources skill was built before the third use case). Dan’s rule is conservative and right — wait until the third invocation before formalizing. Worth applying to the Notion-task-board cycles before they each become “yet another skill.”
Open follow-ups
- Prototype
expertise.ymlfor /process-newsletter. One file, captures sender-by-sender heuristics (subject patterns that predict low/high signal, link-follow patterns, etc.). After 30 newsletter cycles, audit whether the expertise file has converged or is still drifting. ~2 hours to scaffold + 4 weeks of passive data collection. - Add a
self-improvestep to /check-board. Final action of every cycle: append one learned fact to ~/.claude/skills/check-board/expertise.yml. Examples: “Notion data sources don’t accept SQL filters via the workspace search API,” “Sort tiebreaker on identical createdTime should be page-id-ascending.” - Document the expertise-file vs source-of-truth distinction in SOUL.md. One paragraph. Names the two categories. Names the rule (founder utterances are immutable; agent learned patterns are mutable; expertise files are NEVER cited as authoritative in newsletters).
- Audit CLAUDE.md for sprawl — split into core + per-skill expertise files. Estimate: 60% of CLAUDE.md is skill-specific guidance that doesn’t need to load on every invocation. ~3 hours to refactor + test.
- Sanity Check angle: “Your agents forget. Here’s the smallest thing that fixes it.” Lead with the agents-forget problem; introduce expertise files as just-in-time context; contrast against memory-file always-on context; close with the operational discipline (self-improve must be step three). Strong newsletter candidate, especially for the data engineering audience that’s still using AGENTS.md / CLAUDE.md as their only context surface.
- Investigate the “meta-expert” mentioned in passing (~15:00). Dan says he has a meta-prompt that generates expertise files. If the pattern is sound, the same meta-prompt could bootstrap RDCO’s per-skill expertise files automatically rather than hand-writing each one. Tactical brief, ~30 min.
Sponsorship
The video is structured as lesson seven of Tactical Agentic Coding / Agentic Horizon — Dan explicitly references this as a lesson and the closing pitch reinforces the course as the place to “go deeper.” The technical claims (agent experts, expertise files, meta-agentics, Core Four) stand on their own; they are descriptive of patterns Dan has been demoing publicly and are independently testable. The COURSE-AS-CONCLUSION pipeline is the bias to flag: Dan’s framing implies that mastery requires the paid course. Per RDCO’s discipline: extract operational ideas from the public video (free), do not buy the course, and validate the patterns against our own implementation before declaring them effective. The expertise-file claim specifically should be tested in our skill ecology before being cited as authoritative in any Sanity Check issue.
Related
- ~/rdco-vault/06-reference/transcripts/2026-04-20-indydevdan-agent-experts-self-improving-transcript.md — raw transcript
- ~/rdco-vault/06-reference/2026-04-20-indydevdan-one-agent-to-rule-them-all.md — orchestrator commands the experts
- ~/rdco-vault/06-reference/2026-04-20-indydevdan-big-3-super-agent.md — multi-vendor variant; experts are a layer below
- ~/rdco-vault/06-reference/2026-04-19-indydevdan-cracked-claude-agent-skills.md — skills are the static unit; experts add the self-improve loop
- ~/rdco-vault/06-reference/2026-04-19-indydevdan-top-2-percent-plan-2026.md — custom-agents plus private-evals are the umbrella; experts are the operational form
- ~/rdco-vault/06-reference/2026-04-15-thariq-claude-code-session-management-1m-context.md — context-rot principle; just-in-time expertise is a direct mitigation
- ~/rdco-vault/06-reference/2026-04-11-garry-tan-thin-harness-fat-skills.md — expertise files are the “fat” inside skills; harness stays thin