"How Anthropic enables self-service data analytics with Claude" — Anthropic
Why this is in the vault
A first-party Anthropic post describing how they wired Claude into their own data stack to automate self-service business analytics. It is direct evidence for the MAC thesis: the bottleneck on agentic analytics is data-model legibility and test/validation discipline, not model capability. Anthropic's own numbers say curated procedural knowledge ("Skills") plus a semantic-layer-first router is what moved accuracy from unusable to production-grade.
The core argument / what they built
The framing: "enabling self-service business analytics has traditionally been a slog." The two failure modes they call out are (a) overlapping denormalized tables with inconsistent definitions, and (b) ringfenced siloed environments that miss "the long tail of business questions." Their stated insight: "the complexity mainly lies in the ambiguity of the data," not in the tooling.
What they built is a layered agentic analytics stack (no warehouse/BI vendor is named — Snowflake, Databricks, Looker, dbt are NOT mentioned in the post):
- Data Foundations — dimensional modeling, canonical datasets, metadata, freshness checks.
- Sources of Truth — a semantic layer holding metric definitions, plus lineage/transformation graphs, a query corpus, and business context.
- Skills — Markdown procedural-knowledge files the agent reads on-demand, plus structured reference docs. Skills act as a router: the agent is "structurally required" to consult the semantic layer first, falling back to reference docs, which narrows the search space from "a million-field warehouse" to "a few dozen curated files." This is the load-bearing mechanism.
- Validation — offline evals (auto-generated-then-validated dashboard evals + "long tail evals" for uncommon questions, stored as telemetry in a warehouse table), plus online validation: an adversarial-review sub-agent, provenance footers (which source tier the answer came from and how fresh the data is), data-quality checks, and passive monitoring of semantic-layer usage.
Skill maintenance is enforced in CI: skills live in the same repo as the transformation code, and CI flags model changes that don't touch a skill file. Result: ~90% of data-model PRs now include a skill change.
Who uses it: explicitly aimed at "less technical coworkers" and "employees with little desire to learn SQL." Design constraint: results "can't require the user to validate the underlying correctness" because the end user can't.
Hard numbers (as claimed):
- "95% of business analytics queries are automated via Claude, with ~95% accuracy in aggregate" (their words — accuracy is aggregate, not per-query).
- Skills alone moved eval accuracy from ~21% to consistently above 95%.
- Adversarial review adds ~6% accuracy but costs ~32% more tokens and ~72% higher latency.
- ~90% of data PRs now include skill changes; offline eval gate ~90% pass rate per domain before launch.
- NOT given: total user count, query volume, time saved, adoption % among non-technical staff.
Honest caveat (their own): the "silent failure" mode — answer is wrong but plausible and used unchallenged — is acknowledged as unsolved. Provenance footers and sign-off are framed as mitigations, not fixes.
Mapping against Ray Data Co
This is the strongest external validation of the MAC thesis the vault has seen, but it is validation-of-thesis, not a drop-in productizable pattern. Three honest reads:
(a) MAC / audit-model / generate-tests toolchain — direct thesis confirmation. Anthropic's whole stack is an argument that data quality and model legibility are the precondition for self-service/agentic analytics, which is exactly MAC's positioning. Their semantic-layer-first router + canonical datasets is the same "make model behavior legible" move as the MAC 3×6 Scope × Basis matrix ([[2026-04-19-mac-vs-published-data-quality-frameworks]], [[testing-matrix-template]]). Their CI rule (90% of data PRs carry a skill change; skills colocated with transformation code) is the procedural-discipline version of what MAC sells as a contract. The notable gap: Anthropic invested heavily in validation (offline evals + adversarial review + provenance), which MAC's matrix-and-tests framing under-weights. The "silent failure" problem they couldn't solve is precisely the gap MAC's Stop/Pause/Go severity tiers and Human basis are meant to address — that's a sharpening input for MAC, not just a citation.
(b) Founder's phData / Snowflake context — high transfer. Self-service analytics on a governed data estate IS the layer phData consults in. This post is a vendor-agnostic but Snowflake-shaped reference architecture (semantic layer, canonical datasets, lineage, CI-enforced docs) that maps cleanly onto a Cortex Analyst / semantic-view engagement. It's useful as a "here's how the AI-lab itself does it" artifact for client conversations — the 21%→95% delta is a clean talking point for why semantic layer + curated skill docs beat raw text-to-SQL.
(c) Agent-first-interface / agent-orchestration thesis — reinforces it. The Skills-as-router pattern (structurally forcing the agent to consult sources of truth in tier order) is the same instantiation pattern RDCO already uses internally (see [[2026-04-04-anthropic-skills-internally]] and the COO agent's own skill library). It generalizes: the constraint isn't a smarter model, it's curating the few-dozen-file context the agent is required to read first.
Verdict on productizability: this is a validation of the MAC thesis and a usable reference architecture for phdata-style engagements, NOT a ready-made RDCO product. The productizable seed is the validation layer (evals + adversarial review + provenance footer), which is the part MAC's current matrix doesn't yet cover and the part that takes "tests pass" to "the analyst can trust the answer without checking it."
Related
- [[2026-06-03-anthropic-ant-cli-api-command-line-tool]]
- [[2026-04-04-anthropic-skills-internally]]
- [[2026-04-19-mac-vs-published-data-quality-frameworks]]
- [[2026-06-03-claude-api-sdk-capability-map-rdco-use-cases]]