06-reference

data engineering central is data modeling dead

2026-07-27·reference·source: Data Engineering Central·by Daniel Beach
data-modelingsemantic-layeragentic-toolinglakehouse

Why this is in the vault

Daniel Beach's "is data modeling dead" essay is the clearest single-issue steelman-and-rebuttal of the modeling-is-obsolete argument seen from this sender, and it lands two threads (agent-authored pipelines, business-semantic modeling) that map directly onto live RDCO bets.

The core argument

Prompted by a disagreement with a contact ("Andreas," who argues modeling is overrated) set against Joe Reis finishing a new modeling book, Beach asks whether data modeling is dying as a discipline. His answer: no — the perceived decline tracks a tech-stack shift (SQL Server/Kimball-era OLTP-OLAP giving way to lakehouses, Spark, Iceberg/Delta Lake, DuckDB/Polars), not a real drop in modeling's value. He backs this with a small empirical check: a Rust script he wrote scanned 30 scraped Senior Data Engineer job postings for "data modeling" and found it required in 16/30 (53%). An informal poll of his ~20k LinkedIn/~37k Substack audience skewed roughly 80/20 toward "still important," with the dissenting 20% citing tool abstraction, small-data simplicity, and One Big Table/Medallion patterns as reasons modeling gets skipped.

The steelman for "dead": Kimball's Toolkit (the last "bible") is from 1996; the relational-DB-dominant world it assumed is gone; many classic modeling techniques solved performance problems modern engines no longer have; lots of teams run lakehouses with little-to-no modeling and it works fine for them.

Beach's actual position (the centerpiece, quoting Joe Reis directly): "We Model the Business, Not Just Data Systems." The strongest surviving case for modeling isn't a storage-engine argument, it's a semantic one — models exist to encode business context for humans and for "AI trained on humans," and that need doesn't shrink as compute layers change, especially on platforms (Databricks, Snowflake) now exposing data to agentic tooling. He closes with a contingency: small/simple stacks (DuckDB+Polars+Parquet) can "survive but not thrive" without modeling; larger, complex platforms make-or-break on it. He discloses he currently works somewhere running an intentionally old-school Kimball-style facts-and-dimensions lakehouse, and flags that as a possible source of his own prior.

⚠️ Sponsorship

Mid-article paid placement: "Todays sponsor is Estuary." Pitches Estuary Agent Skills — an offering that lets an AI assistant draft production-grade data pipelines from plain-language prompts "right inside Claude Code, Cursor, or the tool you already use," with output as reviewable, versionable config-in-repo rather than a separate agent mode. Clean third-party ad block, no disclosed author/investor relationship visible in this issue. Note for pattern-tracking: SeattleDataGuy (a different K-list author on this same whitelist) has a disclosed adviser relationship with Estuary — this DEC placement is a standard paid slot with no such disclosure, so treat it as a separate, undisclosed-relationship-status sponsor appearance rather than assuming the SDG adviser tie extends to this author.

Mapping against Ray Data Co

Most concrete connection: Estuary's "Agent Skills" pitch — natural-language-to-pipeline, output as reviewable config-in-repo inside Claude Code/Cursor — is structurally the same wager RDCO is running on itself: a skill-based harness (~/.claude/skills/*/SKILL.md, the station-* brigade pattern) where agent output is versioned, reviewable artifacts rather than opaque agent-mode actions. This is competitive-pattern evidence that the "skills as the unit of agent capability" bet is being independently arrived at elsewhere in the data-tooling market, not just an RDCO-specific hypothesis.

Second connection: Beach's "we model the business, not just data systems" (via Joe Reis, an already-tracked K-list author) reinforces the CAF wedge framing — Fabric as a governed knowledge graph exists to encode business semantics for both humans and downstream AI consumers, and the UNOWNED gap (claimed via a port-set spec) is exactly the kind of business-context modeling this essay argues doesn't disappear just because storage engines changed. Worth a light cross-reference next time the CAF Fabric spec is revised: the argument here is independent, non-phData validation of "governed semantic model matters more, not less, as agentic consumers multiply."

Related