Fundamentals of Data Engineering Ch 3 · Designing Good Data Architecture

What is this chapter about?

A company's sales, inventory, ads and support data live in dozens of separate systems. When the boss asks "which channel makes us the most money?", the answer has to be pulled together from all of them. Planning in advance how that data flows, where it lives and who looks after it is data architecture. This chapter is about what makes such a plan "good".

An analogy

Data architecture is like planning a city that never stops growing. A good planner doesn't fix every building on day one. They lay the roads, water and power that everyone needs, then let each neighborhood build what it needs. A good plan lets the city grow without bulldozing the old town every time it expands.

Why the old world was hard

Set in stone, impossible to change. Many companies spent years building one giant system; when the business shifted, the whole thing was out of date.
Pull one thread, the sweater unravels. Parts were so tangled that changing one broke others.
Everything dumped into one big pool. The pile kept growing, nobody knew what was in it or what was still usable, and it turned into a swamp no one dared touch.

How it works, intuitively

Tell apart doors you can walk back through from doors that lock behind you. Make easy-to-undo decisions quickly; think slowly only about the ones you can't undo. Good architecture makes most decisions the first kind.
Assume things will break. Like a city keeping fire crews and backup power, decide ahead of time how fast you must recover and how much data you can afford to lose.
Each neighborhood runs itself and meets the others at agreed crossings. Departments trade data only through fixed "interfaces", so what happens inside one doesn't break the others.
Three generations of storeroom. First a "tidy archive": everything is sorted before it comes in, so lookups are fast, but it only accepts neat things. Then a "junk room": throw anything in first, cheap, but it gets messy. Today's favorite is a "junk room with a librarian": things still sit in the cheap room, but a catalog and rules make it as dependable as the archive.
Central kitchen, or every household cooks? As a company grows, one central team handling all the data becomes a bottleneck. The other idea: each department treats its own data as a "product" and serves it up properly, while the center only supplies the kitchen equipment and shared hygiene rules.

What it buys you / how to choose

With these principles a team can grow alongside the business without starting over. The right choice depends on size: a small team is fine with one off-the-shelf cloud setup; only when there are so many departments that the central team can't keep up is it worth handing data back to each department. One honest cost: the more spread out it is, the more flexible it gets, but the harder it becomes to agree on shared definitions and rules.

One line to remember

Good data architecture isn't a blueprint drawn once; it's a plan that keeps most decisions undoable, keeps the parts loosely joined, and keeps adjusting as the business changes.

Want the book's nine architecture principles, one-way vs two-way doors, the path from warehouse to lake to lakehouse, how to weigh Lambda against Kappa, and the four principles of data mesh? → Switch to Deep

The one-liner

You might think good data architecture is an end-state blueprint with the right tools picked. The chapter defines it instead as the design of systems to support the evolving data needs of an enterprise, achieved by flexible and reversible decisions reached through a careful evaluation of trade-offs. The key word is "reversible": warehouse, lake, lakehouse, Lambda and data mesh are all trade-offs for particular situations, and none of them is the finish line.

Glossary

Data warehouse: a central database built for analytics; data is cleaned and shaped into a fixed structure before it goes in.
Data lake: raw data of every kind (tables, logs, images) stored as-is in cheap object storage and interpreted when it's used.
Object storage: massive, cheap file-style storage such as S3, billed by capacity.
ETL / ELT: Extract, Transform, Load. ETL transforms before loading into the warehouse; ELT loads raw data first and transforms inside the warehouse.
Coupling: how much two components depend on each other. With tight coupling, changing one forces a change in the other.
ACID: four guarantees of database transactions; in plain terms, a write either fully happens or doesn't happen at all, never half.
Multitenancy: many customers or teams sharing one system, with their data kept strictly apart.
RTO / RPO: recovery time objective (how fast you must be back after a failure) and recovery point objective (how much recent data you can afford to lose).

Where it sits

Chapter 3 of Part I, "Foundation and Building Blocks". Chapter 2 listed data architecture as one of the lifecycle's six undercurrents; this chapter gives it room of its own, and Chapter 4 moves on to choosing concrete technologies under these principles. In practice it maps to the architecture review behind "how should we build our data platform, and can we still change it in a few years?"

What problem this chapter tackles

A data team's most expensive mistake is rarely one broken pipeline; it's architecture that locks itself in. An on-premises warehouse that took three years to build can't follow the business when strategy changes. All raw data poured into one lake leaves nobody, three years on, able to say what's usable or which files hold personal data that must be deleted. A single central team queues requests from dozens of departments, and a new report waits a quarter.

The chapter first defines data architecture, then gives nine principles for judging whether it's good, and finally puts the mainstream patterns (data warehouse, data lake, lakehouse, Lambda, Kappa, the unified Dataflow model and data mesh) against the same yardstick. The authors stress that a data engineer need not be an architect but must be architecture-aware, able to tell whether a decision is reversible.

Core ideas, explained

1. What data architecture is: operational, technical, and two kinds of door

Starting from several enterprise architecture definitions (TOGAF, Gartner and others), the authors distill their own: enterprise architecture is the design of systems to support change in the enterprise, achieved by flexible and reversible decisions reached through careful evaluation of trade-offs; data architecture is its data-side subset. It has two layers. Operational architecture answers "what": which business processes to support, how good the data must be, how much latency from creation to query is acceptable. Technical architecture answers "how": how data is ingested, stored, transformed and served. "Move 10 TB per hour from source databases into the lake" is a technical-architecture statement.

"Reversible" borrows Jeff Bezos's framing. A one-way door is a decision that's almost impossible to undo, such as selling the company or hard-wiring your core systems into one proprietary platform. A two-way door can be walked back at any time, such as moving a batch of objects to a cheaper storage class and moving them back if it doesn't work out. Two-way doors deserve speed and volume; only one-way doors deserve slowness and review meetings. Good architecture turns as many decisions as possible into two-way doors.

2. Nine principles, distilled from cloud vendors' frameworks

Drawing on the AWS Well-Architected Framework and Google Cloud's principles for cloud-native architecture, the authors list nine:

Choose common components wisely: share object storage, version control, orchestration and observability across teams instead of reinventing them, but don't force a one-size-fits-all solution that hurts particular teams.
Plan for failure: set availability, reliability, RTO and RPO targets first. A daily executive report may be fine with a 4-hour RTO and a 24-hour RPO; a pipeline feeding online fraud checks may need minutes. Different targets can differ in cost by an order of magnitude.
Architect for scalability: not only scaling up, but scaling down, even to zero when idle, trading elasticity for cost.
Architecture is leadership: architects work mainly by mentoring teams and explaining trade-offs, not by issuing orders.
Always be architecting: architecture is ongoing work; know the current baseline, the target architecture, and the sequencing plan between them.
Build loosely coupled systems: expanded in section 3 below.
Make reversible decisions: technology moves so fast that today's best choice may be obsolete in a few years.
Prioritize security: move from "guard the perimeter, trust the inside" to zero-trust, verifying every access; in the cloud, understand the shared responsibility model, where the provider secures the underlying infrastructure and you secure your configuration and data.
Embrace FinOps: in the cloud, cost shifts from one-off hardware purchases (capital expense) to ongoing pay-per-use spending (operating expense). One bad query or a forgotten cluster can burn a month's budget overnight, so cost must be monitored as closely as performance.

3. Loose coupling: tiers, monoliths, microservices and the strangler

The book cites Amazon's famous internal API mandate: every team must expose its data and functionality through service interfaces, with no direct reads of another team's database. It's the best-known example of loose coupling: each team answers only for its interface and can change its internals without breaking anyone.

In data systems, a single-tier architecture puts database and application on one machine, the simplest and the most fragile; a multitier architecture separates data, application logic and presentation. A monolith keeps everything together, easy to start but hard to change; microservices split it into independent services, easy to change but harder to operate. Factors to weigh: single-tenant or multitenant (multitenancy must guard against noisy neighbors hogging resources, and above all against one customer's data leaking to another); and event-driven design (services talk asynchronously through events, so producers and consumers never wait on each other).

For modernizing old systems the authors recommend the strangler pattern: instead of a big-bang rewrite, the new system takes over the old one's functions piece by piece until the old one can be retired. In effect it splits one huge one-way door into many two-way doors.

4. Warehouse, lake, lakehouse: three generations of analytics architecture

The classic data warehouse definition comes from Bill Inmon: a subject-oriented, integrated, nonvolatile and time-variant collection of data in support of management's decisions. The traditional approach is ETL, cleaning before loading. Cloud warehouses (Amazon Redshift led the way, followed by Google BigQuery and Snowflake) separate compute from storage and scale on demand, which made ELT popular: load raw data first, then transform it with the warehouse's own compute. A slimmed-down subset for one department is a data mart.

The data lake rose in the Hadoop era: pour all raw data into cheap storage first and decide its structure when you use it. The problems came quickly. Without catalogs and governance the lake became a data swamp; data was written and never read; updating or deleting individual records was painful, especially when GDPR required erasing one user's data.

The data lakehouse, coined by Databricks, keeps data in open-format files on object storage but adds a table-format layer on top that provides ACID transactions, schema management, updates and deletes, giving the lake the warehouse's reliability. The authors observe that warehouses and lakes are converging: cloud warehouses can now query files in the lake directly, lakes look more and more like warehouses, and the two are merging into a "data platform".

Three generations: the warehouse tidies before entry, the lake admits first and asks later, the lakehouse puts warehouse transactions and governance back on the lake.

The authors also name the modern data stack: assembling off-the-shelf cloud services as modular, plug-and-play, cost-effective parts. Its value is a lower barrier to entry; its risk is that the number of modules, the bill and the integration work all keep growing.

5. Lambda, Kappa and the unified Dataflow model

What if you need batch and streaming at once? The Lambda architecture (proposed by Nathan Marz) runs two systems in parallel: a batch layer periodically recomputes accurate results, a speed layer uses stream processing for low-latency approximate ones, and a serving layer merges both for queries. Its fatal flaw is that the same business logic must be written twice, once in each system, and when the two disagree it's hard to debug.

The Kappa architecture (proposed by Jay Kreps in 2014) responds by using one stream-processing platform only: all data is an event stream, and recomputing means replaying the log from the start. The authors' verdict is that it hasn't been widely adopted in practice, because storing and replaying huge logs for a long time is complicated and expensive.

Google's Dataflow model (open-sourced as Apache Beam) takes a third path: treat all data as events and aggregate over various kinds of windows. Batch is just a bounded stream, so one codebase runs both. Flink and Spark have adopted similar ideas.

Lambda uses two systems for "accurate" and "fast" at the price of writing logic twice; Kappa and Dataflow treat batch as a bounded stream and keep one codebase.

6. Data mesh: handing data back to the business domains

Most of the patterns above assume one central team owns all the data. Data mesh, proposed by Zhamak Dehghani, rejects exactly that centralization: as a company grows, the central data team becomes a bottleneck and sits too far from the business to understand what the data means. It rests on four principles:

Domain-oriented decentralized data ownership: the orders team owns order data, rather than handing it to a central team.
Data as a product: each domain builds its data as a product for others, with documentation, quality commitments and an owner.
Self-serve data infrastructure as a platform: the central team stops doing data work for others and instead provides infrastructure every domain can use on its own.
Federated computational governance: rules on security, definitions and interoperability are set jointly by domain representatives and enforced automatically in code wherever possible.

It's the same idea as the API mandate in section 3: loose coupling along organizational boundaries. The book also mentions labels such as data fabric and data hub, a reminder that such names keep multiplying; when judging them, go back to the nine principles.

Key trade-offs and choices

Table 1 · Warehouse, lake, lakehouse: what to use when

Architecture	Fits	Cost
Cloud data warehouse	Mostly structured business data, analysts working in SQL; fastest to get going	Awkward for semi-structured and unstructured data; data sits in the vendor's format, so switching platforms is close to a one-way door
Data lake	Huge volumes of logs, images, ML training material; exploratory analysis	Turns into a swamp without governance; updates and deletes are hard, raising compliance risk
Lakehouse	Analytics and ML sharing one copy of data; several engines reading the same table	You operate table formats, metadata and small-file compaction yourself; still evolving fast

Rule of thumb: if your data is mostly tidy business tables and your team works in SQL, start with a cloud warehouse; if ML and unstructured data dominate, or you want to avoid lock-in to one vendor, consider a lakehouse.

Table 2 · Three ways to run batch and streaming together

Approach	Idea	Cost
Lambda	Batch layer for accuracy, speed layer for latency, serving layer merges	Two codebases, two systems to operate; reconciling results is hard
Kappa	Streams only; recompute by replaying the log	Long-term storage and replay of huge logs is costly and complex; not widely adopted
Dataflow / Beam	Batch is a bounded stream; one codebase, two modes	Steep concepts (windows, late data); depends on how well the execution engine supports them

Table 3 · Centralized platform vs data mesh

Organization	Fits	Cost
Central team	A data team of a few to a few dozen people, not many business domains	Becomes a queueing bottleneck at scale; far from the business
Data mesh	Large organizations with many domains able to do their own data work	Every domain needs data engineering skills; definitions and governance are harder to align; heavy platform investment

Why it matters / in real systems

This chapter's vocabulary is the common language of data platform architecture reviews. Asked "why a lakehouse rather than a warehouse?", a good answer doesn't recite tool names; it explains how the data splits between structured and unstructured, whether there are ML use cases, which decisions are one-way doors, what the RTO and RPO are, who watches cost, and whether the organization is big enough to need decentralization. On the warehouse side that means Redshift, BigQuery and Snowflake; on the lakehouse side, open table formats such as Delta Lake and Apache Iceberg; for unified batch and streaming, Apache Beam, Flink and Spark.

Open-source evidence

Delta Lake: an open-source lakehouse. The project describes itself as "an open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive", and notes that its ACID guarantees rest on the atomicity and durability of the underlying storage system. That's Table 1's "add transactions on the lake" in practice. Source: delta-io/delta README, GitHub
Apache Iceberg: many engines safely sharing one table. It calls itself "a high-performance format for huge analytic tables" that "brings the reliability and simplicity of SQL tables to big data, while making it possible for engines like Spark, Trino, Flink, Presto, Hive and Impala to safely work with the same tables, at the same time". That's what decoupling compute from storage, and avoiding lock-in to one engine, looks like. Source: apache/iceberg README, GitHub
Apache Beam: the Dataflow model, open-sourced. It describes itself as "a unified model for defining both batch and streaming data-parallel processing pipelines", with the same pipeline runnable on backends such as Flink, Spark and Google Cloud Dataflow. That's Table 2's "one codebase, two modes" in practice. Source: apache/beam README, GitHub

Pitfalls and debates

"Architecture is a one-time blueprint." The chapter's definition says the opposite: architecture is ongoing work, and baseline, target and transition plan all move with the business.
"Every decision deserves caution." Reviewing two-way doors as if they were one-way doors slows a team until it can't compete; only the few one-way doors deserve slowness.
"The lakehouse killed the warehouse." The two are converging; the choice depends on data shape and team skills, not on which is newer.
"Build the lake now, govern it later." A lake without catalogs, owners and the ability to delete becomes a swamp within a few years, and GDPR erasure requests turn it into a compliance risk.
"Lambda is the standard answer for batch plus streaming." Its two codebases are a long-term burden; most teams today look at a unified model first.
"Data mesh is a technology." It's first an organizational change; without data skills in each domain and agreement on governance, no amount of tooling makes it work, and for small teams it's often over-engineering.
Limits of its time, and debates. The book came out in 2022, while the lakehouse and table-format contest was still under way; since then the formats have moved steadily toward interoperability. How well data mesh works in practice is also disputed, and many companies adopted only its "data as a product" part.

Key takeaways

① Data architecture = supporting evolving data needs through flexible, reversible decisions reached by careful trade-offs; it splits into operational (what) and technical (how) architecture.

② Go slow on one-way doors, fast on two-way doors; good architecture makes as many decisions as possible two-way.

③ Nine principles: choose common components wisely, plan for failure (RTO / RPO), scale up and down, architecture is leadership, always be architecting, loose coupling, reversibility, security first (zero-trust), embrace FinOps.

④ Loose coupling runs from Amazon's API mandate to data mesh; modernize old systems with the strangler pattern, breaking a big one-way door into small two-way ones.

⑤ Warehouses tidy before entry, lakes admit first, lakehouses put ACID and governance back on the lake; warehouses and lakes are converging.

⑥ Lambda's two codebases are hard to reconcile, Kappa relies on replay but saw little adoption, Dataflow / Beam treats batch as a bounded stream.

⑦ Data mesh's four principles: domain ownership, data as a product, self-serve platform, federated governance; it's an organizational problem before a technical one.

⑧ A good architecture-review answer reports data shape, one-way doors, recovery targets, cost and organization size, not tool names.