Data’s Big Whiff — The Case for Elevating Ad Hoc Analysis

Data teams are stuck in a Groundhog Day loop: build dashboards and self-serve tools to escape mundane questions, promise to work on strategic initiatives “later,” but that day never comes.

The core argument

The most important analytical work — the stuff behind one-way-door decisions — is ad hoc analysis, not dashboards. Yet “ad hoc” carries connotations of temporary, unimportant, throwaway work. The term itself devalues the work.

Jeff Bezos’ famous one-way doors are the stuff of ad hoc analysis, not a BI report or self-serve dashboard.

The problem

Ad hoc analysis outputs — the materials containing analyses and recommendations — are scattered across analysts’ computers, buried in emails and Slack posts, built on ungoverned queries and notebooks that blend development work with final recommendations. For work that serves as the intellectual underpinning behind a company’s most important decisions, this is a failure of infrastructure.

The reframe

We need better language, better infrastructure, and more respect for the analytical work that actually drives decisions. The field has no trouble inventing buzzwords (big data, data science, decision science, data mesh) — surely we can dignify the work that matters most.

Connects to analytics as craft, analytics is a mess, why analytics efforts fail, the missing analytics executive, SC E09: one-way/two-way doors.

Open questions

• What would “governed ad hoc analysis” actually look like as tooling?
• Is the real problem that we measure data teams by dashboard output rather than decision quality?