"DS-STAR: How Google built a Data Science agent that actually works" — Jose Parreno Garcia

A secondhand explainer of Google's DS-STAR paper (Google Cloud: Jinsung Yoon, Jiefeng Chen, Raj Sinha, Tomas Pfister + KAIST: Jaehyun Nam; arXiv 2509.21825, submitted Sep 2025, revised to v4 Feb 2026). Founder shared it 2026-06-04 framed as "Google's data science agent architecture." Garcia's piece is the lens; the underlying artifact is a research paper, not a Google product.

Why this is in the vault

DS-STAR is a working reference design for the exact thing RDCO is building toward: a multi-agent system that does open-ended analytics work over messy real-world data and verifies its own output before declaring done. The architecture is a near one-to-one structural cousin of RDCO's own pipeline-* seats and the verification-as-independent-worker pattern. It also lands directly in the founder's phData data-engineering / analytics-agent lane. Worth filing because it's a credible, benchmark-backed validation of the seat-based-loop bet, from a lab, not a vendor deck.

The core argument / architecture

Thesis: single-pass code generation fails on real data-science questions because real data is heterogeneous, multi-file, and the right workflow isn't knowable up front. DS-STAR replaces the single shot with step-by-step planning inside a verify-and-route correction loop. Seven agent roles (claimed in the article):

1. Analyser — deterministically profiles each input file via Python scripts (columns, types, content). Note: profiling is deterministic Python, not LLM-guessed. This matters.
2. Planner — generates one step at a time (init mode for first step, next mode with prior context), not a full upfront plan.
3. Coder — writes executable Python for the current step, holding state across iterations.
4. Debugger — catches execution failures (syntax, missing columns, type mismatch), summarizes the error, generates a fix.
5. Verifier — emits a binary sufficient / insufficient verdict on whether the cumulative plan answers the original question.
6. Router — on "insufficient," decides between Add Step (extend) or Replace Step K (correct a specific prior step), then loops back to Planner.
7. Finaliser — synthesizes outputs into a formatted answer matching the question's guidelines.

Orchestration (control flow): files analysed in parallel up front; then an iterative loop of Planner → Coder → Executor → (Debugger on failure) → Verifier → (Router on insufficient) → back to Planner, terminating on a sufficient verdict or a max-iteration cap. Garcia's framing, worth keeping: "The Verifier is the sensor. The Router is the controller. The Planner and Coder are the actuators." It's a control loop, not a pipeline.

Key novelty vs prior DS agents (claimed): incremental step-at-a-time planning over single-pass; deterministic profiling over LLM file-reading; an explicit correction loop where the Router can replace a specific bad step rather than only appending; handling of heterogeneous multi-file datasets; and a DS-STAR+ variant aimed at open-ended queries that produces research reports rather than single QA answers.

Benchmark numbers (demonstrated in the article):

• DABStep (450 tasks; 72 easy, 378 hard): Gemini 2.5 Pro alone scores 12.70% on hard tasks; DS-STAR + Gemini 2.5 Pro scores 45.24% — a ~32-percentage-point gain. Article's load-bearing claim: the gain comes from system design, not a bigger model.
• KramaBench (104 challenges, 1,700 files, 24 sources, 6 domains): cited for autonomous relevant-file discovery; no headline score quoted in the article.

Claimed vs demonstrated caveat: the 32-point DABStep lift is the one hard demonstrated number. The article reports no competitor baselines (e.g. vs Claude Code or other agent frameworks) and no KramaBench score. Treat "actually works" as one-benchmark-strong, not field-wide-proven. All seven role names and the loop are as Garcia describes the paper; I did not read the arXiv source directly.

Mapping against Ray Data Co

(a) Against RDCO's pipeline- seats + adversarial-verify pattern — strong validation, with one instructive difference.* RDCO's build-out pipeline is four seats: pipeline-spec-author → pipeline-test-author → pipeline-code-author → pipeline-critic, run in a convergence loop. DS-STAR's Planner/Coder/Verifier/Router is the same skeleton: Planner≈spec, Coder≈code-author, Verifier+Router≈the critic-plus-convergence-controller. The shared core bet — a separate seat judges sufficiency and a separate seat decides the next move, neither of which is the producer — is exactly RDCO's "verification belongs to an independent worker" doctrine ([[feedback_verification_independent_worker_pattern]]) and the fresh-eyes-subagent fix for self-preferential bias ([[2026-06-02-thariq-dynamic-workflows-harness-for-every-task]] names that bias explicitly). DS-STAR independently arriving at the same shape from a research direction is corroboration, not just an echo.

Two differences worth stealing or noting:

• Router's Replace-Step-K is more surgical than RDCO's critic. RDCO's pipeline-critic returns PASS/FAIL + per-axis diagnostics and the loop re-runs; DS-STAR's Router explicitly chooses to correct one identified prior step rather than re-plan wholesale. That's a cheaper, more targeted repair primitive RDCO's convergence controller could adopt.
• DS-STAR's Verifier is a single binary verdict; RDCO's critic fans out one subagent per axis in parallel isolation (the structural fix against axis-coupling false patterns). RDCO's design is arguably stronger on the verification side — DS-STAR validates the loop, RDCO's per-axis isolation is the more defensible critic. Don't downgrade to a single binary verifier on this evidence.

(b) Against the founder's phData data-engineering / analytics-agent work — directly transferable. This is the most concrete hook. DS-STAR's premise (messy multi-file client data, no known-good workflow, deterministic profiling first, then iterate) is precisely what client analytics-agent engagements look like. The Analyser-as-deterministic-Python-profiler is a pattern the founder could lift wholesale into a Snowflake/dbt analytics-agent: profile the warehouse schema deterministically before letting an LLM plan, rather than have the model hallucinate columns. The Debugger-on-execution-failure loop maps to the kind of self-healing dbt/SQL test-and-fix loop that /audit-model and /generate-tests gesture at. This is a citeable, lab-backed reference design for "how do we build a data-science agent that doesn't faceplant on real data" — useful in a phData GenAI-Specialty cert / partner-architect context.

(c) Against the agent-orchestration growth lane. Reinforces the house thesis that the win is in the harness/orchestration, not the base model — the article's headline is literally "system design, not the model, drives the gain." Same drum as [[2026-06-02-thariq-dynamic-workflows-harness-for-every-task]] (dynamic workflows / bespoke harness per task) and the broader unhobbling-the-COO-agent direction. DS-STAR is a clean external data point that the orchestration lane is where measurable lift lives.

Related

• [[2026-06-02-thariq-dynamic-workflows-harness-for-every-task]] — Anthropic's dynamic-workflows; same orchestration-over-model thesis, names self-preferential bias that DS-STAR's separate Verifier guards against
• [[2026-06-03-anthropic-self-service-data-analytics-with-claude]] — analytics-agent product framing; DS-STAR is the research-architecture cousin
• [[feedback_verification_independent_worker_pattern]] — RDCO's independent-verifier doctrine that DS-STAR's Verifier/Router corroborates
• [[2026-06-04-a2a-protocols-beyond-mcp-rdco]] — (may not exist yet) agent-to-agent orchestration adjacent