While everyone’s definition of the modern data stack differs slightly (i.e., the tool they sell is the hub around which the whole apparatus spins), there’s little dispute over its general contours. An ingestion tool writes data from a wide variety of sources into a central warehouse; a transformation tool models that data in the warehouse, converting it from raw ores to usable alloys; a BI tool provides direct access to data so that it can be visualized and analyzed. Over the last year, a couple extra complications are becoming popular as well: Reverse ETL tools write data back into source systems, and monitoring tools track the health of the whole system. (View Highlight)
Note: Agreed upon layers
Ingestion
Transformation
BI (visualization & analysis)
Layers picking up steam
“Reverse ETL” (embedded operational analytics)
Monitoring layer - although I believe this sits over each layer
Missing layer
Metrics Layer - author argues this is missing
Though companies use data for a lot of things, one of the most important is also one of the most mundane: basic reporting on business operations. Employees across a company have to make decisions; to make those decisions, they need to know what’s happening. Which products do people like? Which marketing campaigns are attracting new customers? Who on the sales team is hitting their quota? For most companies, data isn’t an AI-powered screenwriter; it’s just a simple narrator, telling people what’s going on. (View Highlight)
Note: Basic reporting is mundane. How do you make that exciting? Similar to how athletes drill the basics
People “want to choose from a list of understood KPIs, apply it to a filtered set of records, and aggregate it by a particular dimension. It’s analytical Mad Libs—show me average order size for orders that used gift cards by month.” (View Highlight)
Note: Self-serve
The core problem is that there’s no central repository for defining a metric. Without that, metric formulas are scattered across tools, buried in hidden dashboards, and recreated, rewritten, and reused with no oversight or guidance. (View Highlight)
Defining metrics in a BI tool localizes those definitions to that tool—or even worse, to individual charts or elements within that tool. A Tableau calculated field is only accessible in the dashboard that uses it; LookML is only accessible in Looker itself. (View Highlight)
