CHAPTER DEEP READ · ACCELERATE · CH 2

Measuring Performance

Accelerate: The Science of Lean Software and DevOps · Ch 2 · Forsgren, Humble & Kim · 2018

What is this chapter about?

You could judge a restaurant kitchen by counting how many plates it washes in a day — but that tells you almost nothing about whether the diners enjoyed their meal. That is exactly the problem Chapter 2 of Accelerate takes on: when you score a software team, what should you actually count? Chapter 1 claimed that a team's delivery ability can be measured. This chapter makes good on that claim: what to measure, how, and why those particular numbers.

Start with something strange

The three yardsticks companies reach for most often — how many lines of code got written, how much work got finished this month, how fully booked everyone is — each make the team worse the more seriously you use them. Count lines, and the code bloats. Count work finished, and the estimates inflate. Book everyone solid, and delivery slows down.

Why the old world was hard

The problem isn't that nobody has found the right yardstick yet. It's that all three share one flaw: they measure how busy you are, not whether anything reached a user.

The most counterintuitive one is "how fully booked everyone is." Booking people solid sounds like plain common sense — idle time is waste, isn't it? But it works like a motorway. When traffic is light, a car arrives and drives straight through. Once the traffic fills the road, one light tap on the brakes up front backs everything up behind. Same with people: fill a calendar to the last minute and anything unplanned has nowhere to go but a queue — and time spent queuing usually dwarfs time spent actually working.

There is a subtler flaw too: each of these yardsticks measures one box. The people writing code are scored on "how many features shipped"; the people keeping the site up are scored on "did anything break." Their optimal moves point in opposite directions. So both sets of numbers look fine, and the user is still waiting.

The idea: count four outcomes instead

The chapter lays down two rules for picking a yardstick, and the rules are worth more than the four numbers they produce:

First, measure the outcome of the whole chain, never one box — otherwise teams are guaranteed to work against each other. Second, measure whether the thing arrived and whether it works, not how busy people were or how much they wrote. Run those two filters and you get four numbers you could explain over dinner: how often you ship, how long it takes from finished code to a user having it, what share of what you ship breaks the site, and how long it takes to recover when it does. The first two say "fast"; the last two say "steady."

Why these four in particular

The trick is that the four numbers pull against each other. To make "ships often" look good you have to automate enough to keep each change small — otherwise "share that breaks the site" turns ugly immediately. Go the other way and the cheapest route to "nothing breaks" is to stop shipping — at which point "how often you ship" hits the floor. Neither side can be made to look good by sacrificing the other. That is the anti-gaming design.

There is a mindset shift buried in there too: asking "how fast do we recover" concedes that failure is unavoidable. The question stops being "how do we never break" and becomes "when we break, how fast are we back."

The honest cost, in one sentence: these four numbers are a health check-up, not a KPI — the moment they're used for rankings or bonuses, people will game them, and gaming them is far easier than improving them.

Remember this one thing

Stop counting how much got done; count whether it arrived and whether it held. Four numbers — how often you ship, how long from finished code to a user having it, what share breaks, how fast you recover — the first two cover fast, the last two cover steady, and read together nobody can win one side by wrecking the other.

Want the operational definitions, the actual magnitudes and the anti-gaming design? → Switch to the deep read

In one sentence

Chapter 2 turns Chapter 1's promise into a concrete measurement design. Its real contribution is not the list of four metrics but the two rules behind the list: measure global outcomes, not local outputs; measure outcomes, not output. You'd assume the hard part is finding the right metric. The hard part is that most metrics turn toxic the moment they become a target — so the chapter first dissects three notoriously poisonous old yardsticks (lines of code, velocity, utilization), then argues why these four numbers constrain one another and can't be gamed in isolation.

Glossary

Software delivery performance — how fast and how reliably a team turns code into working functionality in production; the book treats it as one measurable whole.
Output vs outcome — output is "how much stuff you produced" (lines, tickets, meetings); outcome is "what that stuff caused" (did the feature reach users, is production healthy).
Lead time — a Lean term: the time from a customer making a request to that request being satisfied. The book uses the narrow variant, delivery lead time — from code commit to that code running successfully in production.
MTTR / MTBF — the first is "how long until service is restored after an incident"; the second (mean time between failures) is "how long until the next incident" — and it quietly assumes failures can be prevented.
Batch size — how much change is packed into one delivery. A core Lean variable: smaller batches surface defects earlier and make rework cheaper.
WIP (work in process) — things started but not yet delivered. In a factory it piles up visibly on the shop floor; in software it's unmerged branches, PRs waiting on review, features queued for release — invisible.
Proxy metric — when the thing you actually want to measure is hard to measure, you substitute something easy that moves in the same direction.
Velocity / story points — the relative unit agile teams estimate in, and the points completed per iteration; intended as a capacity-planning tool.
Utilization — how fully people or resources are booked. The unbooked remainder is called slack, and it absorbs unplanned work and improvement work.
Queueing theory — the branch of mathematics that studies waiting lines; the chapter borrows it to explain how high utilization stretches delivery times.
Cluster analysis — a statistical method that lets the data sort itself into groups rather than imposing boundaries in advance.
Feature flag — a switch that turns a feature on or off in production, so "code goes live" and "users can see it" become separate events.
Goodhart's law — the widely cited rule of thumb that once a measure becomes a target, it stops being a good measure, because people start optimizing the measure itself.

Where it sits

Chapter 2 follows directly on Chapter 1, near the top of Part I ("What We Found"). Chapter 1 argues that delivery performance is measurable and predicts organizational performance; this chapter turns "measurable" into an operational definition — the only chapter in the book devoted to how to measure. Its weight comes from what depends on it: every later finding ("capability X improves performance") is stated in terms of the dependent variable defined here. If this chapter's definitions don't hold, the whole book's conclusions go with them. Real-world counterpart: any review meeting that starts "so how is our engineering effectiveness, and what should we fix next quarter," plus that DORA-metrics dashboard on your cloud provider today.

What problem is this chapter solving?

Software is hard to measure for a physical reason: it has no visible inventory. On a factory floor, the machine with work piled up in front of it is the bottleneck, visibly. Software's work-in-process is unmerged branches, PRs stuck in review, features built but not shipped — the bottleneck has no shape. So "what should we count" is never self-evident the way it is on an assembly line; it has to be designed.

Meanwhile the three off-the-shelf yardsticks — lines of code, velocity, utilization — are all toxic, and toxic in the same way. A concrete scenario: at an e-commerce company, the dev team's velocity climbs from 40 points per iteration to 55, and the quarterly slide looks great; on the other side, the change advisory board (CAB) meets once a week and lets through 2 release windows, so finished features sit in the release queue for an average of 3 weeks. Both teams hit their KPIs, and from the user's point of view this company's delivery speed improved by exactly zero.

What happens if you don't fix this? Leadership makes decisions against a room full of green numbers while having no idea whether the company delivers quickly. Worse, those numbers aren't merely useless — they actively push teams toward local optima: dev hoards work to protect its velocity, ops throttles windows to protect its stability record. The metrics themselves become the engine of the conflict.

The core, explained properly

1. Three toxic yardsticks — and the single flaw they share

Lines of code. Reward writing more and you get bloated software, raising both maintenance cost and cost of change. Rewarding writing less doesn't work either: taken to its logical end, the optimum is writing nothing at all. The root problem is that code is a cost, not an output — two implementations of the same feature can differ several-fold in size while being indistinguishable to the user.

Velocity / story points. It is a capacity-planning tool: extrapolate how many iterations this pile of work will take. Used as a productivity measure it has three hard failures. It is relative and team-defined — team A's 5 points and team B's 5 points aren't the same unit, so ranking across teams isn't even arithmetically meaningful. It triggers estimate inflation the instant it's scored — the same job goes from 3 points to 8, the number rises 167%, delivery doesn't move. And teams start hoarding work and refusing to collaborate, because unblocking the team next door doesn't land in your own column.

Utilization. High utilization is only good up to a point. The intuition from queueing theory: the closer utilization gets to 100%, the more steeply lead time climbs, tending toward infinity — with no slack, anything unplanned can only queue, and queueing time grows far faster than linearly. So "keep everyone fully booked," the most conscientious-looking management move available, is precisely what guarantees slower delivery.

Fig 1 · The queueing-theory intuition: the closer utilization gets to 100%, the more steeply lead time rises. Booking people solid isn't efficiency — it's mortgaging delivery time to the queue.

The shared flaw is worth stating on its own: the first two measure output rather than outcome; the third measures local busyness rather than global flow. That diagnosis is more useful than the three examples — you can point it at whatever new metric your company proposes next and tell whether it's another poisonous one.

2. The two rules — the chapter's real methodology

Rule 1: measure a global outcome, not local output. The point is to stop teams being set against each other. The classic pair is dev (wants throughput) and ops (wants stability): as long as their metrics live in two separate boxes, the rational move for each is to undercut the other. A global measure cancels the bet — all four numbers belong to one delivery chain, so whatever one side sacrifices lands on the same ledger.

Rule 2: measure outcomes, not output. Don't reward effort put in. Output — lines written, meetings held, tickets closed — is easy to count and therefore easy to game. Outcomes — did it reach users, does it hold up — are much harder to fake.

Both rules outlive the four metrics. Metrics date: DORA itself added a fifth in 2021. Rules don't.

3. The four metrics, one at a time

(1) Delivery lead time — only the back half is measured. In Lean, lead time runs from a customer request to that request being satisfied. Software splits that span naturally in two: design and validation (a vague idea → a thought-through, buildable plan) and delivery (code committed → running in production). The two halves behave completely differently. The first is exploration by nature — high variability, and squeezing it is a mistake, because pressure there just produces plans nobody thought through. The second should be fast and predictable, and it is the direct expression of engineering capability. So the book measures only the back half: from code committed to that code running successfully in production.

Why short lead time matters: faster feedback makes correction cheaper. Finding out three weeks later that a technical decision was wrong costs tens of times more than finding out three hours later, because by then other code has grown on top of it. Magnitudes: high performers under one hour, low performers between one week and one month (the 2017 State of DevOps figures quoted in the book). The most common trap is the start point: plenty of organizations start the clock when a request enters the backlog, which measures product scheduling and prioritization meetings rather than engineering capability. The two definitions can differ by an order of magnitude, and mixing them leaves you rebuilding the pipeline when what needed fixing was the product decision.

Fig 2 · Lead time splits in two. The exploratory half is supposed to be high-variance; the book measures only "commit → live in production," the half engineering actually controls.

(2) Deployment frequency — a stand-in for batch size. What the authors really want to measure is batch size: how much change is packed into one delivery, since decades of Lean experience say smaller batches flow better. But software has no visible inventory, and "how big is this change" has no common unit across languages and teams — you can't weigh a 300-line backend change against a 30-line frontend change on the same scale. So they settle for the next best thing and use deployment frequency as a proxy for batch size: everyone can report it, definitions mostly agree, variability is low, and the direction is unambiguous — ship more often and each shipment carries less.

Magnitudes: high performers on demand, multiple times per day; low performers between once a week and once a month. Two traps. Deploy ≠ release — only once feature flags separate "code goes live" from "users can see it" is deployment frequency a clean engineering-cadence signal; otherwise product scheduling contaminates it. And a release can be split artificially into ten to inflate the count — which is exactly why it has to be read alongside the other three.

(3) Time to restore service (MTTR). Behind this metric sits a change of assumption. Traditional reliability measurement watches MTBF, mean time between failures, which quietly assumes failures can be prevented. This book assumes failure is unavoidable, so the question becomes how quickly you recover. The downstream effect is large: it is why later chapters put their weight on making rollback fast and safe rather than on stacking up approvals to keep failure outside the door. Magnitudes: high performers under one hour; low performers between one day and one week. The trap is definitional: does "restored" mean users can transact again, or that the root cause is fixed? Those can differ by an order of magnitude — pin the definition down before arguing about the number.

(4) Change fail rate. Definition: of the changes pushed to production (software releases and infrastructure configuration changes alike), what percentage degrades service or causes an outage and subsequently requires remediation — a rollback, a patch, a hotfix, a fix-forward.

Why a percentage rather than a count? Because shipping more often necessarily produces more incidents in absolute terms, and only a ratio compares across cadences. This is the most visible piece of anti-gaming design in the set: it structurally closes off "cut incidents by shipping less" — halve your releases, halve your incidents, and the ratio doesn't budge. Magnitudes: high performers 0–15%; low performers 31–45% (2017). The remaining trap is who defines "failure": narrow it to P0 incidents only and the number instantly halves.

Fig 3 · The four metrics sit on the throughput and stability sides and constrain each other: optimizing either side alone pushes the other over. The anti-gaming property comes from the two rules below.

4. From four numbers to three clusters — and the odd thing about medium performers

Feed the four numbers into cluster analysis, let the data group itself, and you get high, medium and low performers. Two details matter more than the fact that there are three groups:

First, medium and low performers land in the same band on both "fast" metrics (see Table 2). The book's reading is that medium performers are stuck mid-transformation — automation is under way and commits have got more frequent, but legacy systems and heavyweight approvals are still on their back, so the speed dividend hasn't arrived while the risk already has. The most painful stretch of a transformation is usually the middle, not the start.

Second, the 2016 data is sharper still: medium performers had a worse change fail rate than low performers. The plausible explanation is that low performers ship so rarely, and often to less critical systems, that nothing moves and nothing breaks. That isn't stability; it's an absence of delivery. Which is precisely the chapter's design intent proving itself: the four metrics have to be read together, because any single one of them can be read backwards.

Key trade-offs and choices

Table 1 · The four metrics: definition, magnitude, easiest way to game it, how to close that off

Metric	Operational definition (book's version)	High performer	Easiest way to game it → the fix
Delivery lead time	code committed → running in production	< 1 hour	quietly move the start to "entered the backlog" → pin the start at commit; give product scheduling its own metric
Deployment frequency	successful deploys to production (proxy for batch size)	on demand, many per day	split one release into ten → read it with change fail rate; use feature flags to separate deploy from release
Time to restore service	incident → service back to normal	< 1 hour	"restored" as user-facing recovery vs root cause fixed → agree one definition and put it in the postmortem template
Change fail rate	percentage of production changes that degrade service and need remediation	`0–15%`	narrow "failure" to P0 only → fix the definition in your severity scale; using a ratio already closes off "ship less"

Table 2 · The three clusters (2017 State of DevOps figures as quoted in the book)

Metric	High	Medium	Low
Deployment frequency	On demand, multiple per day	Once per week to once per month	Once per week to once per month
Delivery lead time	Less than one hour	One week to one month	One week to one month
Time to restore service	Less than one hour	Less than one day	One day to one week
Change fail rate	`0–15%`	`0–15%`	`31–45%`

Note that medium and low coincide exactly on the first two rows — the "stuck mid-transformation" effect above. The headline multiples between clusters (as quoted in the book: 46× deployment frequency, 440× lead time, 96× recovery, 5× change fail rate) look shocking because the bands themselves span orders of magnitude, not because high performers are working more weekends.

Table 3 · Work out what you actually want to know, then pick the yardstick

The question you're asking	Use this	Not this
Do we deliver fast and reliably?	The four metrics, as your own trend	Cross-team ranking — the definitions differ, so they don't compare
How many iterations will this work take?	Velocity (its actual job)	The four metrics — they don't forecast schedules
How is this individual performing?	Peer review and quality of their work	The four metrics — team-level; applied to a person they will be gamed
Is what we build worth anything?	Product metrics: retention, conversion, task completion	The four metrics — ship 50 useless features a day and all four stay green
What should we fix next?	Take the worst of the four and trace upstream to the constraint	Copying somebody else's list of practices

Why it matters / how it lands in real systems

The four numbers on the engineering dashboards you see today in Google Cloud, GitHub, GitLab and Azure DevOps take their operational definitions from this chapter; the DORA team joined Google Cloud in 2018 and turned them into an open-source measurement stack and an annual report. But what industry actually inherited is the chapter's measurement design method: decide which outcome you want to improve, then pick metrics, then check that the set constrains itself so no single one can be gamed. In interviews and architecture reviews, reciting four terms when asked "how do you measure team performance" is the passing grade; leading with the two rules, then explaining how the four numbers hold each other in check, then volunteering that they don't measure product value — that's the answer of someone who read this chapter properly.

FIELD EVIDENCE · NOT THE BOOK'S CLAIMS — THE COMPANIES' OWN WORDS

Google Cloud turned the four metrics into an open-source project, Four Keys, with definitions lifted almost verbatim from this chapter: Deployment Frequency is "How often an organization successfully releases to production," Lead Time for Changes "The amount of time it takes a commit to get into production," Change Failure Rate "The percentage of deployments causing a failure in production," Time to Restore Service "How long it takes an organization to recover from a failure in production." More telling, they wrote this chapter's "measure both sides together" into a single sentence: "At a high level, Deployment Frequency and Lead Time for Changes measure velocity, while Change Failure Rate and Time to Restore Service measure stability."Google Cloud Blog, "Four Keys," 2020-09-22 ↗
The metric set grows and the thresholds drift — the rules don't. In 2021 DORA publicly conceded it had been measuring availability rather than reliability, and expanded the set: "Historically we have measured availability rather than reliability, but because availability is a specific focus of reliability engineering, we've expanded our measure to reliability so that availability, latency, performance, and scalability are more broadly represented." The same report already sorts teams into four clusters rather than three (elite was added), with "elite performers… increasing their lead time for changes from less than one day to less than one hour" — the 2017 table in the book has thresholds that are lenient by today's standards.Google Cloud Blog, "DORA 2021" ↗
DORA itself argues against using the metrics for cross-company ranking: the official line in the 2023 edition is "Comparing your metrics year-over-year is more meaningful than comparing yourself to other companies." These four numbers are for measuring yourself against your own past, not for seating teams by rank.Google Cloud Blog, "2023 State of DevOps Report" ↗
This chapter's lead author drew the boundary around her own four metrics three years later. Nicole Forsgren and colleagues introduced the SPACE framework in ACM Queue, stating: "Developer productivity is about more than an individual's activity levels or the efficiency of the engineering systems relied on to ship software, and it cannot be measured by a single metric or dimension." Delivery performance is one facet of effectiveness, not the whole of it.Forsgren et al., "The SPACE of Developer Productivity," ACM Queue 19(1), 2021 ↗

Common mistakes and open arguments

The biggest misreading: treating it as a measure of product value. The four metrics measure delivery capability — you can ship 50 features a day that nobody uses and keep all four green. Value needs product metrics (retention, conversion) measured separately. Conflate the two and you conclude "our performance is great but the business isn't moving, so these metrics are useless," which is wrong about the metrics.
Team-level diagnosis ≠ individual performance review. Applied to a person it triggers Goodhart's law immediately; cross-team ranking doesn't hold up either, since definitions and system criticality differ. DORA's own 2023 position is that year-over-year comparison against yourself is what's meaningful.
The data is self-reported and the definitions were never uniform. "What counts as a deployment" and "what counts as a failure" get different answers at different companies, so cross-company comparability is much weaker than those headline multiples suggest. The soundest use is your own time series.
The lead-time start point is still argued about. The book pins it at commit, but plenty of organizations have their real bottleneck upstream of commit (request queues, prioritization meetings). Use only the book's definition and you get a flattering number plus an undiscovered bottleneck — measure both spans, just don't blend them into one figure.
The thresholds are of their time. From 2018 DORA moved to four clusters (adding elite); by 2021 the elite lead-time threshold had moved from under a day to under an hour. Reliability was only added in 2021, and security, code quality and developer experience still aren't in these four numbers. Grading yourself against the 2017 table is grading yourself against the industry of seven or eight years ago.
"Velocity is useless" is an overreading. The chapter objects to velocity as a productivity score, not to its value for capacity planning. Likewise, "utilization is harmful" means "being fully booked is harmful," not "lower is always better" — the argument is for slack, not for idleness.
The four metrics constrain each other, but they aren't jointly ungameable. Narrow the definition of "failure" and split one release into several, and you can in principle raise deployment frequency and lower change fail rate at the same time. The real defense isn't in the metric design; it's not making them a KPI — and that is the layer the book offers no solution for.

Takeaways

① This chapter turns Chapter 1's "performance is measurable" into an operational definition, and is the only chapter about how to measure; every later finding rests on the dependent variable defined here.

② The physical reason software is hard to measure: no visible inventory — work-in-process is unmerged branches and release queues, so the bottleneck has no shape.

③ Three toxic old yardsticks: lines of code (mistakes cost for value), velocity (a capacity-planning tool turned into a score, producing estimate inflation and work hoarding), utilization (queueing theory: the closer to 100%, the steeper lead time climbs).

④ The two rules outlive the four metrics: measure global outcomes, not local output (cancel the dev-vs-ops bet), and measure outcomes, not output (don't reward busyness).

⑤ Delivery lead time measures only commit → live in production (the exploratory half is supposed to vary; the start point is the classic trap); deployment frequency is a proxy for batch size (batch size can't be measured directly, this can, with low variance and the right direction).

⑥ Time to restore service encodes a change of assumption: from MTBF (how long until failure) to failure is unavoidable, so how fast do we recover — which is why later chapters favor fast rollback over more approvals.

⑦ Change fail rate is a ratio, not a count, structurally closing off "cut incidents by shipping less" — the most visible piece of anti-gaming design in the set.

⑧ Memorize the magnitudes (2017): high performers deploy many times a day, with lead time and restore time both < 1 hour and a 0–15% fail rate; medium and low coincide on the two speed metrics, which says the painful stretch of a transformation is the middle.

⑨ The two things to stay wary of: the metrics measure delivery capability, not product value; and they are a team-level diagnostic, not an individual KPI — DORA's own guidance is that comparing against your own last year is what's meaningful.