06-reference

dedp dep intro

Fri Apr 03 2026 20:00:00 GMT-0400 (Eastern Daylight Time) ·book-chapter ·source: https://www.dedp.online/part-2/5-dep/_data-engineering-patterns.html ·by DEDP / Simon Späti

DEDP 5.1 — Data Engineering Patterns (Intro)

The framing chapter for DEPs — the middle layer of the CE → Pattern → Design Pattern hierarchy. DEPs are “generally repeated, identifiable designs or practices” in data engineering. They sit between raw convergent evolutions (specific tools) and prescriptive design patterns (best-practice implementations).

The Questions DEPs Answer

The chapter frames DEPs through practical questions practitioners face daily:

QuestionPattern AreaRelated Technologies
How do I cache data for BI?Cachingdbt, materialized views, OLAP, data virtualization
How do I orchestrate pipelines?OrchestrationCron, Python frameworks, Airflow, custom solutions
How do I manage master data without duplication?MDM / Data IntegrationFuzzy joins, entity resolution, golden records
How do I package my data stack?Workspace PackagingDocker, Kubernetes, cloud platforms
How do I share data across teams?Data SharingFile exports, delta live tables, open standards
How do I combine stateful and stateless assets?Asset IntegrationInfrastructure as Code, declarative data stacks

Each question points to a pattern; each pattern has multiple convergent evolution implementations. This is the practical value of the framework — it gives you a name for what you are doing, which means you can find prior art, compare approaches, and avoid reinventing solutions.

Position in the Book

This chapter is a gateway. The specific DEPs covered in subsequent chapters:

Why DEPs Matter for Consulting

For 01-projects/phdata/index engagements, DEPs are the conversation layer. Clients do not think in design patterns — they think in problems. “How do I make my dashboards faster?” is a caching question. “How do I let teams work independently?” is a workspace packaging question. Naming the pattern unlocks the design pattern, which unlocks the implementation.

This is 06-reference/concepts/analytics-as-craft at the architectural level: craft is knowing what pattern you are implementing and choosing the right instantiation, rather than reaching for the tool you happen to know.

Connections