Harvest, Yield, and Scalable Tolerant Systems

In one sentence

In 1999 Fox and Brewer argued that a large service's "availability" shouldn't be a black-and-white switch but should split into two measurable, tradeable quantities — yield (the fraction of requests completed) and harvest (the fraction of data reflected in a single answer). This turns "what to do under failure" from "hold up or fall over" into an explicit choice: for the same failure, you can lower harvest (answer, but incompletely) or lower yield (answer fewer, but fully), degrading gracefully and staying as available as possible through faults and partitions. This short paper is also the first written statement of the "strong consistency and high availability can't coexist" trade-off — distilled a year later into the CAP theorem.

Glossary

Where it sits

The authors are Armando Fox and Eric Brewer, both at UC Berkeley; the piece is a position paper at the 1999 HotOS-VII workshop — short, aimed at proposing a framework rather than running experiments. Brewer had earlier co-founded the search company Inktomi and lived the engineering of large cluster services, and the paper's examples are mostly framed around a search cluster. It inherits the authors' earlier cluster-services work (the 1997 Cluster-Based Scalable Network Services that introduced the BASE idea) and launches Brewer's CAP conjecture at PODC 2000 and the 2002 Gilbert–Lynch formal proof — this is the foundation stone right before CAP.

The problem and the motivation

By the late 1990s internet services were exploding in scale, stacking hundreds to thousands of machines, and failure went from "exception" to "constant" — some fraction is always broken. The old ways of measuring reliability were ill-fitting in two places:

• Treating "available" as binary: the system is either online or offline. But a service of a thousand machines lives almost permanently in the middle state — "mostly fine, a few broken" — which a binary description can't capture.
• Measuring availability as uptime: uptime weights all moments equally, yet failures cluster at peak load (heaviest load is when things break), and a crash then loses far more requests than one at 3 a.m. Uptime systematically underweights the cost of peak-time failure.

The deeper wall: the authors point out that strong consistency (all replicas always in agreement) and high availability cannot both hold when the network may partition. Once a partition splits the cluster into two halves that can't talk, you either let both halves keep serving (give up consistency, each writing on its own) or stop one half to preserve consistency (give up availability). This isn't sloppy engineering — it's a hard trade-off. Since you can't have both, and failure is constant, the authors argue: rather than chasing "never fail," design for "how to degrade gracefully when you do."

The core idea

1. Split availability into two quantities: yield and harvest

Yield = requests completed ÷ requests received. Its difference from uptime is that it's weighted by requests: a crash at peak has more requests in the denominator, so yield drops harder — exactly reflecting "peak failure is costlier." Intuition: yield answers "how many did you actually answer."

Harvest = data reflected in the answer ÷ complete data. It admits that "an answer can be partial": a search that skipped 5% of the corpus but still returns results from the other 95% has harvest = 95%. Intuition: harvest answers "how complete was your answer."

The big payoff of splitting them: failure no longer has only "answer" and "don't answer" as outcomes — a whole buffer of "answer, but less completely" opens up in between, which you can spend to keep "still answering."

2. The DQ principle: capacity is conserved, forcing a choice

The authors offer an empirical rule: near saturation, the product of data-per-request D and requests-per-second Q is roughly constant — capped by the system's physical capacity (memory bandwidth, disk, network). Written D × Q ≈ constant, in plain terms: "enough data and enough requests are the same cake — one grows only as the other shrinks."

This rule translates failure into one plain sentence: a failure that takes x% of capacity is a failure that cuts DQ by x%. Whether you take that cut out of D or out of Q is your choice.

• Cut Q (lower yield): accept fewer requests — even actively reject some — and answer the accepted ones fully, preserving harvest. Done e.g. via cost-based admission control.
• Cut D (lower harvest): accept every request, but put less data into each answer, returning a partial result and preserving yield.

The point: DQ forces you to admit "degradation is unavoidable"; the real design freedom left is "which way to degrade" — and that can be aligned with the business (do you want completeness, or an answer for everyone?).

3. Replication vs partitioning: it decides which quantity you lose

For the same "one node dies" failure, how the data is laid out decides whether you lose yield or harvest:

• Replication: every machine holds the full dataset. Lose one, and the rest still have complete data — harvest doesn't drop, but you've lost one machine's throughput, so yield / capacity drops.
• Partitioning: the data is split up. Lose one, and that slice is gone — harvest drops (the answer is missing that piece), but requests-per-second is nearly unchanged, so yield doesn't drop.

The authors add a counterintuitive observation: near saturation, replication doesn't conjure extra capacity — because DQ is capped by a physical ceiling, replication and partitioning have the same total DQ. What replication actually buys isn't "more capacity" but "preserving harvest under failure." So replication vs partitioning isn't "which is faster" — it's "which quantity you'd rather keep."

4. Decompose into orthogonal subsystems, pen strong consistency into the smallest cage

The last move is architectural: split the application into mutually orthogonal, independently-failing subsystems, each with its own harvest/yield policy; and pen the "must be strongly consistent, must be persistent" hard state into the smallest piece possible, making the rest soft state (rebuildable, tolerant of stale values, eventually consistent). Then one failure hits only a small piece, and most of the system keeps serving with high availability. The authors stress that mechanisms should be orthogonal and composable — state and failures both stay local, so nothing pulls the whole system down.

Key results

This is a HotOS position paper: there are no benchmark tables — its "result" is a way of thinking that endures, plus corroboration from real systems, not experimental numbers. The highlights:

• Introduces harvest / yield as two measurable dimensions, turning "availability" from a binary switch into a continuum.
• Introduces the DQ principle, giving the actionable view "a failure = cutting DQ, and you choose where."
• Uses replication vs partitioning to make clear the mapping to "keep harvest or keep yield."
• Argues from large services such as Inktomi's search cluster: when some nodes fail, returning a partial result (lower harvest) beats being wholly unavailable — this kind of "graceful degradation" was already running in commercial search of the day.

(The paper is mainly argument and design principle; no specific percentages or benchmark numbers are invented here.)

Why it mattered

• The direct ancestor of CAP: the "consistency vs availability can't coexist under partition" laid out here became Brewer's CAP conjecture at PODC 2000, proved a theorem by Gilbert and Lynch in 2002. Nearly every distributed-systems textbook's trade-off chapter starts here.
• It gave "graceful degradation" a theoretical name: modern large services shed precision under overload/failure, return partial results, rate-limit to protect the core — engineering instincts this paper recast as reasoned design choices.
• A path toward eventual consistency and NoSQL: penning strong consistency into the smallest range and making the rest soft state is exactly the philosophy of Dynamo-era eventually-consistent NoSQL; the harvest/yield language also foreshadows SRE measuring services by "success-rate SLO" rather than uptime.

Limitations and criticism

• Harvest/yield never became common vocabulary: good as the concepts are, the industry's daily words are CAP, "N nines" of availability, and SLOs — few actually say "harvest/yield." It's more an illuminating lens than an adopted standard.
• DQ is only an approximation: it assumes near-saturation and homogeneous capacity; it doesn't hold at low load, with heterogeneous requests, or when cache-hit rates vary widely, and D itself is hard to measure precisely.
• CAP was later called too coarse: Brewer himself, in 2012's "CAP Twelve Years Later," reflected that "two of three" is misleading — partitions are actually rare, and the trade-off you weigh daily is "consistency vs availability during a partition" plus consistency vs latency in normal operation; Abadi's PACELC is the completion of it.
• Not every application can lower harvest: a partial/stale answer is fine for search but wrong for an account balance or an inventory decrement — better to answer nothing. The framework fits "soft" use cases best.
• Informal: as a position paper it offers intuition and a framework, not proofs; the rigor came later with the CAP formalization.

The essentials