06-reference

dedp design patterns intro

Fri Apr 03 2026 20:00:00 GMT-0400 (Eastern Daylight Time) ·book-chapter ·source: https://www.dedp.online/part-2/6-dedp/_data-engineering-design-patterns.html ·by DEDP / Simon Späti

DEDP 6.1 — Data Engineering Design Patterns (Intro)

This chapter draws the line between Data Engineering Patterns (DEPs) and Data Engineering Design Patterns (DEDPs). DEPs are immediately implementable building blocks — caching, reusability, workspace packaging. DEDPs are higher-level architectural blueprints: they tell you how to compose those building blocks into systems that solve recurring strategic problems.

The Core Definition

A DEDP is a “general repeatable solution to a commonly occurring problem” — borrowed from software design pattern vocabulary, applied to data engineering’s specific challenges. These are not tool recommendations. They are structural decisions about data flow, modeling, governance, and platform integration.

What DEDPs Address

Five strategic concerns keep appearing across data orgs:

  1. Data flow — how information moves through systems (ingestion, transformation, serving)
  2. Data modeling — organizing data into logical layers that serve multiple consumers
  3. Platform integration — unifying analytics and data science workloads on shared infrastructure
  4. Governance — managing data as discoverable, trusted, reusable assets
  5. Pipeline design — intent-based (declarative) rather than implementation-based (imperative) definitions

These map directly to the kinds of questions 01-projects/phdata/index clients ask: “How should we organize our warehouse layers?” (modeling), “How do we expose metrics consistently?” (governance + flow), “Should we consolidate on one platform?” (integration).

The Named DEDPs

The chapter previews several patterns explored in detail later:

Why This Framing Matters

DEDPs transcend hype cycles. Tools come and go — Informatica to dbt to whatever is next — but the structural problems remain. The pattern vocabulary gives you a way to evaluate new tools against timeless requirements rather than getting swept into trend-based decisions.

This is the 06-reference/concepts/systems-over-goals principle applied to data architecture: design for the recurring problem, not the tool of the moment.

Connections