Continuous Delivery Ch 7 · The Commit Stage

What is this chapter about?

The app on your phone is built by programmers who hand in their "homework" to a shared codebase several times a day. Chapter 7 of Continuous Delivery is about what kind of check should run in the first few minutes after that homework arrives. That check is called the commit stage. It is the first gate of the pipeline from the previous chapter, and the only gate every developer faces personally, every single day.

An analogy

Think of hospital triage. When you walk into the emergency room, a nurse spends five minutes on your temperature, blood pressure and pulse — no scans, no surgery. One job only: use the fastest, cheapest instruments available to spot who is obviously in trouble. The expensive examinations come later, and only for the people triage lets through.

Here is the counterintuitive part: the value of triage is not in how thorough it is, but in being fast enough that you are willing to stand there and wait for the result. A very thorough triage desk that takes two hours is no triage desk at all.

Why the old world was hard

This check can fail in three ways, all fatal. Too slow: pile every check together and it takes two hours, so nobody waits — you hand in your work and move on, and when the report comes back red you have long forgotten what you changed. Useless: the report says only "failed", not what broke or how to reproduce it, so people quietly learn to ignore it. Dishonest: some checks pass and fail at random on identical code; after a few rounds, the team's first reaction to a red light stops being "go fix it" and becomes "just run it again" — and from that moment the gate is dead.

Three rules that fix it

One: give it a time budget of ten minutes. That number is not arbitrary — it is roughly the limit of how long a person will stand there and wait. Past it, behaviour changes: people stop waiting, keep working, and several people's changes pile up so nobody can tell whose change turned the light red. Ten minutes is a design constraint, not a performance target.

Two: only run checks that are both fast and trustworthy. What is slow? Touching a database, clicking through a user interface, waiting a while before looking. Most of this chapter teaches you how to keep those three out: give the little piece of code under test a stand-in — a fake database, a clock that stops on command — so it finishes in a thousandth of a second, and thousands of them together still take seconds.

Three: once it passes, seal the box and label it. That exact box travels down the rest of the line; nobody is allowed to build a new one. The reports get filed alongside it, where the whole team can read them.

The genuinely hard part

The difficulty is not in the checking — it is in the code being checked. For a small piece of code to be tested on its own with stand-ins, it has to be separable from everything around it. With tangled legacy code you will find you simply cannot write fast tests. Which is this chapter's hidden lesson: when tests are hard to write, that is usually not a testing problem — it is the design raising an alarm.

It does have a cost: this gate is fast precisely because it never touches a real database or a real network, so "green" only means nothing obviously broke — it does not mean the change is fit to ship. That verdict belongs to the slower, more expensive gates further down.

One line to remember

The commit stage is the pipeline's first gate: a trustworthy red or green within ten minutes. It does not aim to be thorough — it aims to be fast enough that you wait, and reliable enough that you believe it. Thoroughness is somebody else's job, further down the line.

Want the actual contents, where the ten minutes goes, and how to pick test doubles? → Switch to the deep read

In one sentence

The commit stage is the first gate of the deployment pipeline: triggered automatically by every commit, it compiles the code, runs the commit tests, performs static analysis, assembles a deployable binary, and returns a red or green verdict within minutes. It is the one gate every developer faces personally every day, so the real subject of this chapter is not what to test but how to make this gate fast enough that people wait for it and trustworthy enough that people believe it — ten minutes is not a performance target but a behavioural constraint: past it, developers stop waiting, and the whole discipline of continuous integration unravels behind them.

Glossary

Commit — pushing a change into version control (e.g. Git); it gets a unique revision number, and it is the commit stage's input.
Build / binary (artifact) — compiling and packaging source into something runnable: a jar, a container image, an installer.
Artifact repository — the store for binaries and reports, each tagged with the commit it came from.
Commit tests — the tests that run in the commit stage, overwhelmingly unit tests: they exercise one small piece of logic, touch no database, network or UI, and finish in milliseconds.
Static analysis — checks and metrics computed by reading the code without running it: suspicious constructs, duplication, cyclomatic complexity (how many branch paths a piece of code has), coverage.
Code coverage — the share of lines or branches actually executed while the tests ran.
Test double — a fake collaborator used in a test. Three common kinds: a stub (returns canned values), a mock (also asserts that you called it, and how often), and a fake (a lightweight but genuinely working implementation, such as an in-memory database).
Dependency injection (DI) — an object receives its collaborators from outside instead of new-ing them itself, which is what makes swapping in doubles possible.
Flaky test — a test that passes and fails on identical code, usually broken by time, concurrency or shared state.
Smoke test — a handful of checks run right after deployment, answering only "is this thing alive?".
Build master — in very large teams, a rotating role that watches the build, chases red lights and maintains the build scripts.
Stop the line — from the Toyota production line: when something goes red, halt and fix it rather than letting defective work flow onward.

Where it sits

This chapter follows Ch5 inside Part II. Ch5 drew the whole conveyor belt from commit to production; this chapter zooms into its first cell. Upstream it inherits the discipline of Ch3 (merge to trunk daily, stop the line when red) and the quadrants of Ch4 (the commit stage carries the technology-facing tests that support development); downstream, Ch8 takes over with automated acceptance testing. In today's world it is the thing you already stare at: the required checks on a GitHub pull request, the first stage of a GitLab CI pipeline, a Gerrit pre-submit verify, Google's TAP, Meta's Sandcastle. Those few minutes of spinner you wait on every day are this chapter.

The problem it solves

Ch5 already established that earlier gates must be faster. What it did not say is what this particular gate should look like — and this gate has a special status: it is the only daily interface between developers and the entire pipeline. Most developers may not look at acceptance tests, capacity tests or the production release for a week at a time; they meet the commit stage several times a day. So when this gate is unpleasant, what gets bypassed is not the gate — it is the whole practice of continuous delivery.

The chapter names three concrete failure modes:

Slow. Do the arithmetic: eight developers committing about twenty times a day. At eight minutes per run, people can sit and wait for the result. At forty minutes, you burn thirteen hours of machine time a day, work queues up, and nobody waits forty minutes — they start the next change instead. By the time the light goes red, three or four people's commits are stacked on top of it, and "whose change broke it" is itself an investigation.
Useless. Feedback that says only "FAILED" — not which test, with what input, or how to reproduce it locally. The chapter is explicit that feedback must be actionable: the failure should point straight at the code to look at.
Untrustworthy. Once "it went red, just re-run it" becomes the team's instinct, red has lost its meaning — and a red light with no meaning is the same as no gate at all.

Leave this unsolved and you fall back into the world of Ch1: defects sit in the codebase for weeks, by which time their author has lost all context and diagnosis costs several times more; or the mirror image, a team dragged under by a two-hour suite that flakes, until everyone quietly agrees to route around it.

The core content, properly explained

What the commit stage actually does: one input, two exits

The definition is clean. Input: one commit in version control. Work: compile → run the commit tests → run static analysis → assemble deployable binaries (and, where useful, create database schemas or build installers). Exits: exactly two. Either red — notify the whole team immediately and, per Ch3, stop the line and fix it; or green — store the binaries, reports and metadata in the artifact repository as a release candidate that moves on to the acceptance stage.

Of those three outputs, the one most often neglected is the metadata: which commit this binary came from, which tests it ran, and how they went. It is what every later stage uses to decide on promotion, and the thread you follow backwards when something blows up.

Fig 1 · One input, two exits. Red stops the line; green seals binaries, reports and metadata together — the metadata is what every later promotion decision rests on.

The ten-minute budget: a constraint on people, not machines

The chapter keeps returning to one number: the commit stage should finish in under ten minutes (the same figure the CI discipline of Ch3 gives). Why ten rather than twenty? Because the threshold governs human behaviour, not hardware:

Under ten minutes → developers watch for the result, fix red immediately, context still in their heads;
Over ten minutes → they move on to the next thing, fixes get deferred, later commits stack up, and one failure has to be traced through several people's work;
Longer still → the team starts batching changes up before committing — and larger batches are the source of every pain in Ch1.

This is also why the chapter refuses to put "all the tests" here: the commit stage is not trying to prove the change is right, it is using the cheapest means available to keep the obviously wrong out.

The arithmetic: why commit tests must not touch databases, UIs or waits

A pure in-memory unit test runs in milliseconds. A test that connects to a database, creates and cleans tables easily costs 50-100 ms each. A test driving a real user interface costs seconds to tens of seconds. Multiply by volume and the gap is orders of magnitude:

Fig 2 · Where the budget goes: let every test touch a real database and five thousand of them eat most of the ten minutes; drive the UI and you are out of the running entirely.

Hence a very concrete set of writing rules, all serving one goal — drive the unit price of a test down to milliseconds:

Avoid the GUI. UI tests are slow, brittle and sensitive to rendering timing; they belong to the acceptance stage of Ch8.
Avoid the database. Separate business logic from persistence and substitute the repository layer with an in-memory implementation or a double; keep a small set of genuine integration tests for the real database and run them in a later stage.
Avoid asynchrony. Crossing threads or processes, or "waiting a bit and then asserting", is the single largest source of flakiness. Cut the asynchronous boundary: test the business logic synchronously and the orchestration separately. Where it is truly unavoidable, wrap the wait in polling with a timeout rather than scattering fixed sleep calls — the chapter treats such brute-force approaches as a last resort.
Fake time. Never let code read the system clock directly; make "what time is it" injectable. Otherwise month-end settlement or 30-day expiry logic is either untestable or only red on certain days.
Minimise state. Shared globals and leftover table rows make failures depend on execution order — and such tests are worse than no tests, because they consume trust.

Test doubles and dependency injection: making one small piece testable alone

All of the above presumes one thing: the code under test can have its collaborators swapped out. That is what dependency injection buys you — collaborators arrive through the constructor or the parameters, so a test can hand it a double instead.

Fig 3 · Dependency injection plus test doubles: the same business code talks to a real database at runtime and to in-memory doubles under test, taking a single test from hundreds of milliseconds down to microseconds.

The chapter adds a warning about doubles that has aged extremely well: overuse them and your tests get welded to your implementation. A stub only feeds values back, so you are still testing "is the output right"; a mock asserts "you must call these methods in this order" — refactor the internals, even with behaviour unchanged, and such tests go red in droves. Tests exist so you dare to change the code; when they make you afraid to change it, they have turned into their own opposite.

What should break the build

Compilation failures and failed commit tests are red beyond argument. What is arguable is static analysis: coverage below a threshold, duplication over a limit, complexity spiking, new warnings — should those fail the build?

The chapter's position is yes, and often they should — encoding a threshold the team agreed on into the build, enforced automatically, beats writing it on a wiki nobody reads. But with an important caveat: these metrics are indicators of a trend, not goals in themselves. Coverage is the dangerous one: it tells you which code was never executed, not whether the executed part is well tested — delete every assertion and coverage does not move. So use it to find obvious gaps (a new module at zero), not as a KPI to drive upward.

The outputs: artifact repository, reports, metadata

What comes out of a green run must be kept properly — this is where Ch5's "build your binaries only once" lands. Binaries do not go into version control: they are derived from source, and stuffing them into Git only bloats the repository. Put them in a dedicated artifact repository, named by revision, so you can always trace a package running in production back to its commit. Reports (test results, coverage, static analysis) are first-class outputs too, and belong somewhere the whole team can open them — making analysis visible has value on its own, even before you use it to fail builds.

Who looks after the gate

The last principle is often skipped and is the most organisational of them: the commit stage belongs to the development team, not to a "build team". The people changing the code are the ones who know what checks to add and what a red light means; hand the scripts to a group that writes no application code and the scripts will inevitably drift behind the code, while developers stop feeling that red is their problem. The companion practice is treating build scripts as first-class code: in version control, refactored, reviewed. Very large teams can appoint a rotating build master to watch red lights, chase fixes and tend the scripts — rotating on purpose, to share the load rather than outsource the responsibility.

Key trade-offs

Table 1 · Does this check belong in the commit stage, or later?

Check	Typical cost	Catches	Where	Price / rationale
Compile & package	seconds to 1 min	changes that do not even build	Commit stage	uncontroversial, the cheapest gate there is
Unit tests	milliseconds each	regressions in business logic	Commit stage	requires injectable code; expensive to retrofit onto legacy
Static analysis	seconds to minutes	suspect constructs, runaway complexity and duplication	Commit stage	thresholds set too tight produce daily false alarms, and people learn to ignore them
A few smoke-level integration checks	tens of seconds	components that do not connect, misconfiguration	Commit stage (strictly capped)	the category most prone to creep — cap it deliberately
Database integration tests	`50-100` ms each	SQL, mapping and migration errors	Secondary or acceptance stage	putting them here trades budget for fidelity
End-to-end / UI tests	seconds each and up	a whole business flow that does not work	Acceptance stage (Ch8)	slow and brittle; they will sink the commit stage
Performance / capacity tests	hours	throughput and latency regressions	A later dedicated stage	needs baselines and a dedicated environment

Table 2 · What turns the build red, and what only warns

Signal	Recommendation	Why	Risk
Compile or test failure	Hard red	behaviour has been falsified; nothing to discuss	—
Flaky test	Quarantine with a deadline	leaving it in teaches the team to ignore red	the quarantine becomes a dumping ground and defects escape
Coverage below threshold	Can be red; set by the team, to prevent backsliding	good at spotting a whole new module with no tests	chased as a KPI it produces assertion-free tests
Duplication / complexity limits	Make visible first, enforce once stable	indicators of a trend, not targets	enforced from day one it just breeds evasive coding
New compiler warnings	Hard red (baseline the existing ones)	warnings that cost nothing accumulate forever	with a large backlog, a blanket rule paralyses the team

Table 3 · What commit tests do about the database

Approach	Cost per test	Fidelity	Where	Price
No database at all (logic + doubles)	microseconds to ms	low: SQL and mapping untested	The bulk of the commit stage	demands a clean split between logic and persistence
In-memory database	`~1` ms	medium: dialect differs from the real thing	Commit stage, cautiously	wherever the dialects differ, you get false green
Real database in a container	tens of ms plus startup	high	Secondary / acceptance stage	expensive in 2010, cheap today — the budget still binds
One shared test database	unpredictable	medium	Not recommended	teams pollute each other; failures depend on order

Table 4 · Picking a test double

Double	What it does	Good for	Price
Stub	returns canned answers when asked	feeding inputs: clocks, config, read-only queries	almost none — the default choice
Mock	asserts it was called, how often, with what	verifying side effects: charging, emailing, publishing	welds tests to the implementation; refactors go red in droves
Fake	a lightweight working implementation (in-memory repository)	anything stateful; usually the most natural to write	it is code too — it must be maintained and can drift from the real thing
The real object	no substitution	pure computation with no I/O	the moment I/O appears, speed and stability collapse

Table 5 · The commit stage broke ten minutes. Now what?

Option	Effect	Price / precondition
Shard across machines in parallel	near-linear speedup, the most direct fix	tests must share no state; costs hardware
Move slow tests to a secondary stage	puts the commit stage back inside budget immediately	slower feedback, and what moves out tends to be neglected
Split the application and its pipelines	smaller scope per component	introduces cross-component version combinations (Ch13)
Select tests by impact	run only what this change could affect	needs a dependency graph or build system; a wrong selection lets defects through
Delete duplicated, low-value tests	often a surprising win	someone must actually look at the data rather than guess
Just raise the limit to 30 minutes	looks like the easy way out	people stop waiting, and the CI discipline dissolves with them

All five tables rest on one criterion: every second of the commit stage must be bought with "how much earlier does this check find a problem?" A check that is both slow and only catches rare problems does not belong in this gate — but do not throw it away either; move it one stage down.

Why it matters / in real systems

The shapes changed; the logic did not. The required checks blocking a GitHub merge button, the first stage of a GitLab CI pipeline, a Gerrit pre-submit verify, Bazel's small tests with remote caching, Google's TAP, Meta's Sandcastle — all are engineered commit stages. Artifact repositories (Nexus, Artifactory, container registries) exist for exactly the "binaries plus metadata" output described here. Merge queues supply the piece the book never developed: with many people committing concurrently, each change is re-tested on the post-merge state, so two individually green changes cannot combine into red.

In interviews and architecture reviews this is close to a standard question. Asked "how does your CI work", the answer that scores is not a list of tools but four facts: how many minutes is your commit stage? what turns it red? how do you handle flaky tests, and how many are quarantined right now? when it goes red, who owns it and how long until it is green?

EVIDENCE FROM INDUSTRY · not the book's claims, but companies' own public statements

Google codified "small tests" as a hard rule back in 2010: they must run in a single process, with no network access, no disk access, and no sleeping or blocking calls — an independent rediscovery of this chapter's "avoid the database, avoid asynchrony", written down as engineering policy. Google Testing Blog, "Test Sizes", 2010 ↗
Google later published its target mix in Software Engineering at Google: roughly 80% narrow-scoped unit tests, 15% medium integration tests, 5% end-to-end — this chapter's "put the weight in the commit stage", turned into a number you can measure yourself against. Winters, Manshreck & Wright, Software Engineering at Google, Ch11, 2020 ↗
Google at scale: its CI platform TAP runs about 4.2 million tests continuously and roughly 150 million test executions per day (about 35 runs per test per day), with 99% of executions passing — proof that "run it on every commit" survives extreme scale, at the price of keeping each test extremely cheap. J. Micco, "The State of Continuous Integration Testing @Google", ICST 2017 keynote ↗
The counter-example · what flakiness really costs: Google publicly acknowledged in 2016 that roughly one in seven (nearly 16%) of its 4.2 million tests exhibited some degree of flakiness, forcing dedicated machinery to detect, quarantine and re-run them — that is the price tag on this chapter's "a red light you cannot trust is no red light at all". Google Testing Blog, "Flaky Tests at Google and How We Mitigate Them", 2016 ↗
Meta (Facebook) published the modern answer to "the commit stage no longer fits": use machine learning to predict which tests a change could affect, run about a third of them, and still catch more than 99.9% of regressions before they reach trunk — doubling the efficiency of its testing infrastructure within a year. Engineering at Meta, "Predictive test selection", 2018 ↗

Pitfalls and controversies

Treating the commit stage as the complete test suite. The classic death: every escaped defect adds another batch of tests, and ten minutes slowly becomes forty. The duration is a budget you defend, not an outcome you observe — before adding, ask how much earlier this check would find a problem.
Coverage as a KPI. It only tells you which code ran. Driving it up usually produces assertion-free tests: pretty number, same defects. Use it to find gaps, not to rank people.
Curing flaky tests with automatic retries. Retrying is a painkiller: it hides genuine intermittent defects in concurrency and timing along with the noise. Current practice is quarantine plus a deadline plus a tracked flakiness rate — not infinite retries.
Overusing mocks. Heavy mocking asserts "how it is implemented" rather than "whether the behaviour is right", so refactoring goes red in droves and the team becomes more afraid to change code — the heart of the classicist-versus-mockist argument to this day.
Period limitations. The book is from 2010, before containers, Testcontainers, remote build caches and ML-driven test selection. "Never touch a database in the commit stage" can be relaxed slightly today, since spinning up a real one got cheap; but the ten-minute budget has not moved at all, because it governs people rather than machines. The book also says almost nothing about concurrent commits interfering with each other — exactly what merge queues address now.
Controversy: how large should the unit-test share be? Google's 80/15/5 is one position; another camp argues that excessive unit testing locks in implementations and that real confidence comes from the integration level. The honest answer is that the ratio depends on how testable your architecture is and which layer your defects come from — but there is no disagreement that the commit stage must be carried by tests fast enough to run in milliseconds.

Takeaways

1 · The commit stage is the pipeline's first gate and the only one every developer faces daily — its experience decides whether the whole delivery discipline survives.

2 · One input (a commit), two exits: red stops the line, or green emits binaries + reports + metadata into the artifact repository as a release candidate.

3 · Ten minutes is a behavioural constraint, not a performance target: past it people stop waiting, and both blame attribution and batch size go out of control.

4 · The arithmetic of the budget: unit tests in milliseconds, database tests in hundreds of milliseconds, UI tests in seconds — let each test touch a real database and five thousand of them consume most of your budget.

5 · Writing rules for commit tests: avoid the GUI, avoid the database, avoid asynchrony, fake time, minimise state — all enabled by injecting doubles for collaborators.

6 · Use doubles sparingly: stubs for inputs, mocks for side effects; overused mocks weld tests to the implementation and turn refactoring red.

7 · What should be red: compile and test failures always; static-analysis thresholds may be, but coverage is an indicator, not a goal — it only shows which code never ran.

8 · Flakiness is this gate's number one killer: a red light you cannot trust is no red light at all; Google has acknowledged roughly one in seven of its tests showing some flakiness.

9 · The gate belongs to the development team, not a build team; build scripts are first-class code, and very large teams use a rotating build master.

10 · When it no longer fits: parallelise, split, move work later, select by impact (Meta runs a third of its tests and still catches 99.9% of regressions) — not raise the budget from ten minutes to thirty.