06-reference/research

data quality acceptance frameworks vs mac

2026-05-11·research-brief·source: deep-research

Where MAC Fits in the Data-Quality Acceptance-Framework Landscape (May 2026 refresh)

The question

What published data-quality acceptance frameworks exist beyond Great Expectations and dbt-expectations (Soda, Monte Carlo, Datafold, Anomalo, Bigeye), and how do their "acceptance criteria" definitions compare structurally to MAC's 3x6 Scope x Basis matrix?

This brief refreshes [[2026-04-19-mac-vs-published-data-quality-frameworks]] with three follow-ups the April pass left open: (1) Coalesce Quality (formerly SYNQ, acquired March 2026) and Sifflet, (2) the Wang and Strong 1996 academic framework as a possible structural precedent, and (3) a sharper "novel vs one-of-N" verdict for the founder.

What MAC actually is (correcting the question's framing)

The question describes MAC as "3x6 Scope (Row / Aggregate / Transformation) x Basis (Absolute / Relative / Temporal / Human-sanity-check / + 2 more)." The canonical artifact at [[../../01-projects/data-quality-framework/testing-matrix-template]] is structured slightly differently after the multiplicative refactor:

"Transformation" is no longer a Scope value. It got promoted into the Relative:Source basis (completeness + precision + fidelity across layers) when the founder realized delta is a dimension, not a category. That refactor is the MAC framework's load-bearing structural move and is documented in [[../2026-03-30-founder-data-quality-framework]].

This matters for the comparison: vendors don't have a "transformation" scope either. The thing MAC tests with Relative:Source is exactly the thing Datafold ships as data-diff. The framing where Transformation is a peer scope obscures that overlap.

What we know from the vault

Vendor-by-vendor comparison

The April brief covered the seven main vendors. This refresh re-checks them at 2026-05 and adds Sifflet and Coalesce Quality.

Vendor Acceptance-criteria structure Maps to MAC how?
Great Expectations Flat library of 300+ named "Expectations" (expect_column_values_to_*, expect_table_*). Naming convention groups by scope (column / multi-column / table) but no published basis dimension. Covers Column-Absolute and Aggregate-Absolute densely. Sparse on everything Relative or Temporal. No Human cell.
dbt-expectations + dbt native tests Four generic tests (unique, not_null, accepted_values, relationships) + ported GE library. severity: warn|error config is the closest dbt analog to MAC's Stop/Pause/Go tiering. Same coverage shape as GE plus dbt 1.8 unit tests (synthetic data → Row-Absolute cell). Severity tiers map cleanly to MAC's Stop/Pause/Go but lack the layer-aware remap.
Soda (SodaCL) YAML DSL, ~25-50 built-in metrics. Lives in Git next to transformations. Soda Cloud adds data contracts on top. Flat library of declarative checks per dataset. No published 2D taxonomy. Strong on Column-Absolute, Aggregate-Temporal.
Monte Carlo "Five pillars" marketing taxonomy: freshness, volume, schema, distribution, lineage. ML-discovered monitors, no rules required for baseline. Now extended to "Data + AI Observability" (model inputs, agent behavior, output drift). Five pillars is essentially Aggregate x Temporal repeated four ways plus lineage. Doesn't surface a structural cross-product.
Datafold Primary primitive is data-diff — compare table A vs table B across environments or before/after a PR. Plus column-level profiling and anomaly detection. CI/CD focus. Datafold's diff IS the Relative:Source cell at every Scope level. The only vendor with first-class Relative:Source coverage. Still no Relative:Recon, Temporal, or Human structure.
Anomalo "No rules to define." Unsupervised ML across full datasets — distribution shifts, pattern changes, full table scans. Anti-taxonomy by design. Concentrates monitoring in Aggregate x Temporal. The pitch is that you don't author tests, the system does.
Bigeye Auto-monitor creation per table, 70+ pre-built metrics, ML-suggested thresholds. Five-pillar coverage matches MC. Their 2022 "four data quality categories" blog is a flat market-segmentation list (Observability / Transformation / Testing / Lineage), not a test-design matrix. Same as Monte Carlo for Scope x Basis coverage. Five pillars + ML thresholds.
Sifflet Four-pillar monitor taxonomy: freshness, volume, schema, distribution. Sentinel agent auto-recommends monitors based on risk, change frequency, downstream impact. Same flat four-pillar shape as MC and Bigeye. Adds business-aware lineage and impact analysis. No 2D test-design matrix.
Coalesce Quality (formerly SYNQ, acquired Mar 2026) AI-powered observability, now integrated into Coalesce's "data operating layer" alongside transformation and catalog. SYNQ's pre-acquisition pitch was "detect issues early, understand impact instantly, recover fast." Public docs do not surface a published 2D taxonomy. The strategic move is bundling observability with transformation in one tool, not restructuring how acceptance criteria get authored.

The five-pillar consensus is real and unanimous. Monte Carlo, Bigeye, Sifflet, and (operationally) Coalesce Quality all converge on freshness / volume / schema / distribution / lineage. This is a marketing taxonomy that maps to "what an ML monitor can auto-detect from cardinality and time series." It is not a test-design discipline.

Where MAC's 18 cells map to vendor coverage (refreshed)

Cell Vendor coverage
Column x Absolute Universal — commodity
Column x Rel:Source Datafold (column diffs); GE (cross-table expectations)
Column x Rel:Production Sparse — manual SQL
Column x Rel:Recon Essentially none
Column x Temporal Strong — every vendor's distribution / null-rate monitor
Column x Human None
Row x Absolute GE / Soda / dbt-expectations multi-column rules; dbt 1.8 unit tests covers Row-Absolute with synthetic data
Row x Rel:Source Datafold row-diff
Row x Rel:Production Sparse
Row x Rel:Recon Manual
Row x Temporal Sparse — vendors trend aggregates, not row-level changes
Row x Human None
Aggregate x Absolute Some (Soda metric checks); often hand-coded
Aggregate x Rel:Source Datafold; manual elsewhere
Aggregate x Rel:Production Manual SQL — no vendor primitive
Aggregate x Rel:Recon Manual SQL — no vendor primitive
Aggregate x Temporal Universal — the "five pillars" core
Aggregate x Human None

Vendors are dense in the Temporal column (especially Aggregate-Temporal) and the Column-Absolute cell. They are sparse-to-absent across the four reconciliation cells (Rel:Production and Rel:Recon at any scope), the Row-Temporal cell, and the entire Human column.

The academic precedent (Wang and Strong 1996)

Wang and Strong, "Beyond Accuracy: What Data Quality Means to Data Consumers" is the most-cited DQ taxonomy in the literature. Empirical, derived from 355 data-consumer surveys, organizes 15 dimensions into four categories:

Important: this is NOT structurally analogous to MAC. Wang/Strong is a one-dimensional list of properties data should have (categorized for explanatory purposes). MAC is a two-dimensional matrix of test-design axes — where you check x what you check against. They answer different questions:

That said, the vault note for the MAC content series should cite Wang/Strong as the academic anchor — it gives MAC scholarly cover and is structurally distinct enough that nobody can claim MAC is just renaming the 1996 framework. The vendor landscape descends from Wang/Strong intellectually (the five pillars look a lot like Contextual + Intrinsic dimensions stripped down) but nobody has gone the other direction — turning the dimensions into a test-authoring matrix.

Convergences and contradictions (refreshed)

Where the landscape converges:

Where the landscape diverges:

Synthesis for RDCO — the honest verdict

MAC is structurally novel within the test-design axis, and it is one of N within the broader category. The exact split:

The load-bearing differentiator is the 2D matrix, not the cells themselves. Cell-by-cell, MAC's primitives are mostly available somewhere in the vendor landscape. What is novel is forcing test design through 18 categories rather than through whatever the tool's default check-library happens to be. This is the same move Kimball's bus matrix made for dimensional modeling: it doesn't add new primitives, it makes the coverage gaps visible. That is genuinely original framing in the data-quality space as of 2026-05.

Three RDCO-specific opinions the vendor landscape doesn't ship:

  1. Reconciliation cells are first-class. Rel:Production and Rel:Recon are the cells where consulting work actually lives — tying the warehouse to the finance ledger, to Stripe, to the bank. The vendor landscape treats these as custom integrations. MAC treats them as required cells. This is the explicit "Definition of Done" wedge for the Client Reporting series.
  2. Human Sanity Check is named. The vendor landscape pretends quality is fully automatable. MAC names the unautomatable cell so teams don't drop it once they "have tests now."
  3. Layer-aware severity. MAC's bronze/silver/gold severity remapping has no direct analog in any vendor product. Soda has tags, dbt has warn|error, Monte Carlo has alert routing — but none of them prescribe the same check has different severity at different medallion layers. This is an opinion vendors don't have, and opinions sell consulting.

Repositioning recommendation (unchanged from April, sharpened): Lead the MAC content series with "the matrix the vendors don't ship." Frame Soda, Monte Carlo, Anomalo, Sifflet, Coalesce Quality as partial coverage of MAC — each covers some cells well, none covers all 18. Position MAC as the build-time coverage discipline that determines what to ask any of those tools to monitor — not as a replacement for them. The four cells that vendors structurally can't sell (the three reconciliation cells + Human) are the strongest evidence MAC is filling a real gap, not relabeling existing checks.

Open follow-ups

  1. Has anyone published the "vendor coverage of MAC's 18 cells" comparison as a public artifact? If not, that comparison table above is a free-standing newsletter unit and the strongest single piece of MAC marketing collateral I can imagine.
  2. dbt 1.8 unit tests with synthetic data — confirm in practice that they live in Row-Absolute and don't accidentally cover other cells. If they overlap Row-Rel:Source (synthetic input → expected output IS a transformation check), that's an interesting nuance worth a vault note.
  3. Worth a single-vendor deep dive on Coalesce Quality post-acquisition (mid-2026) to see if SYNQ's roadmap changes the structural story or just the bundling.
  4. Should MAC explicitly cite Wang/Strong 1996 in the canonical artifact? It anchors the framework in 30 years of academic literature and makes it harder to dismiss as "founder-invented." Cost is one paragraph; benefit is credibility for skeptical reviewers.
  5. The "five pillars" (freshness/volume/schema/distribution/lineage) are now industry consensus across 5+ vendors. Worth a short Sanity Check piece on why that consensus is structurally incomplete — it gives MAC a platform to introduce the 2D matrix framing.

Sources

Vault:

Web: