BOOKS DEEP-READ · DDIA · CH 1
Designing Data-Intensive Applications · Ch 1 · Martin Kleppmann · 2017
Every app you use — a shop, a chat, a video site — runs on a system that stores and retrieves data. Chapter 1 of DDIA (Designing Data-Intensive Applications) doesn't teach any one technology. It asks a more basic question first: what makes a data system "good"? The author offers three yardsticks — reliable, scalable, maintainable — and the whole book revolves around them.
Think of a data system as a restaurant. Reliable: "even if a cook calls in sick or one stove breaks, guests still get fed." Scalable: "we handle 10 tables today and don't fall apart when we go viral and 100 turn up next month." Maintainable: "the kitchen flow is clean, so onboarding a new hire or adding a dish isn't a nightmare." This chapter turns those three fuzzy senses of "good" into things you can measure and argue about.
Most people measure speed by the average: 100 ms on average sounds fine. But averages lie — if 99 of 100 requests are quick and one stalls for 5 seconds, the average still looks great, yet the person waiting 5 seconds is often your most important customer (lots of data, heavy request).
This chapter teaches a better question: line up 100 requests fastest-to-slowest and look at the 50th, 95th, 99th. The 99th (called "p99") captures "the experience of the unluckiest slice" — and reputation usually rides on that, not the average.
A part breaking is a fault; the whole system going dark for users is a failure. The trick to a good system isn't "parts never break" (impossible) — it's making sure one broken part doesn't drag down the whole by keeping spares that take over. Netflix even keeps a "trouble monkey" (Chaos Monkey) that randomly kills live machines, forcing engineers to build systems where "losing one box is a non-event."
After this chapter you hold a shared vocabulary for judgment: for reliability, talk about "keeping faults from becoming failures"; for speed, talk p95/p99 instead of averages; for scale, first ask "where exactly is the load growing?" Everything later — replication, partitioning, transactions — is really trade-offs among these three yardsticks. There's no best, only best-fit.
Whether a data system is good comes down to three yardsticks: reliable (a broken part doesn't stop the whole), scalable (it copes when load grows), maintainable (easy to change and run). And stop fooling yourself with the "average" — watch the experience of "the slowest slice" (p99).
Want the mechanisms, notation and diagrams? → switch to Deep
DDIA Chapter 1 teaches no specific technology; it erects the book's thinking framework: the quality of a "data-intensive" system rests on three words — reliability, scalability, maintainability. It turns those fuzzy notions into measurable, tradeable engineering metrics — above all, discussing performance via response-time percentiles rather than averages — setting the yardsticks for every trade-off in the next 11 chapters.
p50 is the median.Author Martin Kleppmann (a Cambridge researcher, ex-LinkedIn engineer). This chapter opens Part I, "Foundations of Data Systems," and is the through-line for the whole book: it doesn't follow from a specific technology — it sets the yardsticks first. Every later chapter — replication, partitioning, transactions, consensus, batch/stream — is a concrete trade-off among these three. In effect, this chapter is the coordinate system for understanding every trade-off that follows.
Engineers casually say a system must be highly available, high-performance, scalable — but those words are slippery: how reliable counts as reliable? Is "fast" the average, or is the slowest also fast? Scale what? Without a shared, quantified vocabulary, teams talk past each other and decide by gut. This chapter's job is to break the three slogans into concepts you can measure, discuss, and trade off — so that when you later debate "single-leader vs leaderless replication," everyone shares a basis for judgment.
The definition is plain: the system keeps working correctly even when things go wrong. The key is separating two words: a fault is a single component deviating from its spec (a disk dies, a process crashes, the network hiccups); a failure is the system as a whole stopping service. Faults are nearly unavoidable, but you can keep them from escalating into failures — that's being fault-tolerant.
The author sorts faults into three kinds. Hardware faults: a disk's mean time to failure is ~10–50 years, so with many machines something breaks almost daily — the classic answer is redundancy (RAID, dual power, hot spares), with the trend shifting toward software-level tolerance. Software faults are more insidious, because they're often correlated, mass outbreaks of systematic errors (all nodes crash on the same trigger; one leap-second bug takes out a fleet). Human faults — configuration errors are the number-one cause of outages. The remedy isn't demanding "no mistakes," but designing for tolerance: decoupling, fast rollback, thorough monitoring, and — like Netflix's Chaos Monkey — deliberately injecting faults into production to flush out fragility.
Table 1 · The three fault classes — character and countermeasures (DDIA's taxonomy)
| Fault type | Typical examples & magnitude | Correlated? | Main countermeasures |
|---|---|---|---|
| Hardware | Disk failure, memory errors, power loss. Disk MTTF ~10–50 yrs → a 10,000-disk cluster loses ~1 disk/day on average | Mostly independent | Redundancy (RAID, dual power, hot spares); trend toward software-level tolerance |
| Software | All nodes crash on the same trigger; one leap-second bug fells a fleet; cascading failures | Highly correlated, mass outbreaks | Decouple, rate-limit, isolate; thorough testing & monitoring; self-healing processes |
| Human | Configuration errors — the #1 cause of outages at large internet services | Correlated (one slip hits many places) | Design for tolerance: decoupling, fast rollback, staged rollouts, monitoring, drills |
Scalability isn't a yes/no label; it must answer concretely: as load grows, by what means does the system keep its performance? Two steps.
Step one, describe the load with a few load parameters — requests per second, read/write ratio, concurrent users, cache hit rate. The book's classic example is Twitter's home timeline: the real load parameter isn't "tweet rate" but the distribution of followers per user — a celebrity's single tweet must reach tens of millions, so fan-out is where the difficulty lives. DDIA's cited figures make the point: posting tweets runs ~4.6k req/s on average and 12k+ at peak, while home-timeline reads hit 300k req/s — a read:write ratio of ~65:1, the pressure on the read side, and every tweet must fan out to all of the author's followers.
Table 2 · Twitter home timeline: magnitude of the two fan-out strategies (figures as cited by DDIA, ~2012 Twitter)
| fan-out on write | fan-out on read | |
|---|---|---|
| Post one tweet | Write into every follower's inbox cache | Write the author's box once |
| Read timeline | Read your own inbox — fastest | Query all followees, merge & sort — slower |
| Write amplification | 1 tweet × ~75 followers avg ≈ 345k writes/s (4.6k×75) | None |
| A celebrity's tweet | 30M+ writes, and must land promptly | 1 write; cost pushed to read |
| Best for | Ordinary users with few followers | Mega-celebrities followed by millions |
Twitter's final answer is a hybrid: the vast majority go fan-out-on-write, a tiny number of mega-celebrities go fan-out-on-read and get merged in at read time — routing by the load parameter (follower distribution) itself. A live demonstration that scalability has no universal answer, only per-load tailoring.
Step two, describe performance — and here is the chapter's most important lesson: don't use the average, use percentiles. Response time isn't a single number but a distribution — most requests fast, a few slow. The average is skewed by a few extremes and hides who is suffering. The right move is percentiles: p50 (median, half of users are faster), p95, p99, p99.9. The tail (p99 and beyond) often matters most, because the slowest requests tend to come from the users with the most data — the most valuable ones. Amazon sets service goals on p99.9, not the average.
Table 3 · How to read response-time percentiles, and what they cost
| Percentile | Meaning | Typical use / cost |
|---|---|---|
p50 median | Half of users are faster | The "typical" experience |
p95 | Slowest 1 in 20 requests | Common SLA target |
p99 | Slowest 1 in 100 | Where the watched tail starts |
p99.9 | Slowest 1 in 1,000 — often the users with the most data, the most valuable | Amazon sets internal SLAs on it |
p99.99 | Slowest 1 in 10,000 | Amazon judged optimizing this far not worth it — diminishing returns, steep cost |
Why are tail latencies stubborn? First, queueing delay often dominates the tail: a server can handle only so many requests in parallel, so one slow request stalls everyone queued behind it (head-of-line blocking). Second, tail latency amplification: if a page fans out to several backends in parallel and must wait for the slowest, even a small slow rate per backend gets magnified at the page level. Put a number on it: if one user request hits 100 backends in parallel and each has a 1% chance of being slow, the odds the whole page is dragged by at least one slow call reach 1 − 0.99¹⁰⁰ ≈ 63% — each backend looks healthy, yet the page-level tail is grim (exactly the effect Google's Jeff Dean and Luiz Barroso quantified in The Tail at Scale). So measure on the client side, by percentile, continuously — not just server-side averages.
Most software cost isn't in building but in long-term upkeep. The author splits it into three: operability — make it easy for ops to keep the system running well (good monitoring, docs, automation); simplicity — manage complexity, using good abstractions to remove accidental complexity (complexity from the implementation, not inherent to the problem), so the system never rots into a big ball of mud; evolvability — make it easy to change as requirements shift. Together they decide whether a system can be kept alive, long-term, without pain.
Table 4 · Scale up vs scale out
| Scale up | Scale out | |
|---|---|---|
| Approach | A stronger single machine (more CPU / RAM, faster disks) | Many commodity machines in a shared-nothing cluster |
| Ceiling | A hard ceiling — even the biggest box has a limit | Near-linear; scales to very large sizes |
| Cost curve | High-end machines cost super-linearly | Commodity boxes in bulk; low unit cost |
| Complexity | Low — the app barely changes | High — brings replication / partitioning / consistency, the whole distributed problem set (Part II) |
| Best for | Non-extreme load, keeping it simple | Large load / high availability, willing to take on distributed complexity |
In practice you often mix both: a few strong machines are frequently simpler and cheaper than a swarm of small ones. There's no magic scaling sauce — architecture is always tailored to the app's specific load.
Countless backend engineers treat this chapter as the common bedrock for interviews and architecture because it hands you a cross-technology vocabulary for judgment: whether you use PostgreSQL, Cassandra, Kafka, or something homegrown, reliability reduces to "did the tolerance boundary contain the fault," performance to "what's p99, where does the tail come from," scale to "what's the load parameter, how do we add machines." Percentile latency (p99/p99.9) is now the standard language of SLAs and monitoring dashboards everywhere; chaos engineering spread from Netflix into common practice. Every later chapter's technology can be judged back against these three yardsticks.
p99.9 (not the average), reasoning that the slowest requests tend to come from the customers with the most data — the most valuable ones — validating this chapter's "watch the tail, not the average." Dynamo paper, SOSP 2007 ↗① One line: the book's through-line — use reliable / scalable / maintainable as three yardsticks to turn "is this system good?" into measurable, tradeable engineering questions.
② Reliability: separate fault (a component breaks) from failure (the whole stops); aim for tolerance — don't let faults escalate; faults are hardware / software / human, with human config errors the most common.
③ Scalability in two steps: describe load with load parameters (in the Twitter case, follower distribution / fan-out), then describe performance.
④ The core lesson on measuring performance: use percentiles, not averages — p50/p95/p99/p99.9; the tail matters most because the slowest users are often the most valuable.
⑤ Tail latency is stubborn due to queueing / head-of-line blocking and tail amplification; measure client-side, by percentile, continuously.
⑥ Ways to scale: up (bigger machine, has a ceiling) vs out (shared-nothing, grows big but adds distributed pain); no magic scaling sauce.
⑦ Maintainability = operability + simplicity (kill accidental complexity, avoid the big ball of mud) + evolvability.
⑧ Significance: it provides a cross-technology vocabulary — the coordinate system for every later trade-off in replication / partitioning / transactions / consensus.