BOOK DEEP READ · CHAOS ENGINEERING · CH 11
Chaos Engineering: System Resiliency in Practice · Ch 11 · People in the Loop · 2020
Earlier chapters showed that big companies deliberately break things in their own systems — switch off a server, slow down the network — to see whether everything holds up. That practice is called chaos engineering.
The obvious next step seems to be: if it's so useful, let the computer do it automatically, every day, at random, and take people out of it. This chapter says: not so fast. The most valuable part of breaking things on purpose is the people.
A hospital can buy a machine that draws blood and prints a report for everyone. But "why test for this", "when to test" and "does this slightly high number actually matter" — those are the doctor's calls.
Remove the doctor and keep only the machine, and you get a pile of numbers nobody understands. The machine handles "how to test"; people handle "why, when, and what it means".
Engineers often feel that people are slow and make mistakes, so anything that can be automated should be, and fewer people means more safety.
But here is the odd thing: once machines do all the routine work, people rarely touch the system and gradually forget what it looks like. Then something happens that the machines have never seen, and it is exactly those out-of-practice people who must save the day. The more complete the automation, the less familiar people are at the moment that matters most.
The gain: drills are no longer machines running on their own; each one updates people's picture of the system, so the next real outage finds them knowing where to look.
One honest cost: people's time is limited, and if every drill needs someone to plan, watch and interpret it, you can't run many — so familiar routines can be repeated by machines, and only new questions deserve human attention.
Chaos engineering is not "let machines break things automatically"; it is letting people use controlled breakage to update their understanding of the system — machines handle "how", people handle "why, when, and what it means".
Want the ironies of automation, a human/machine split table and what real companies do? → Switch to Deep
You think the end state of chaos engineering is "fully automated, unattended"; in fact its output is not a list of failures but a more accurate model of the system in people's heads. The why, how and when of an experiment are human judgments; automation should take over only the repeatable part of the "how", not take people out of the loop.
Chapter 11, the third chapter of Part III "Human Factors", written by John Allspaw (former CTO of Etsy and a leading voice in resilience engineering). It follows Ch10, which made the organisation the subject of experiments, and precedes Ch12, which uses an algorithm to decide where to inject faults; this chapter sits between them and asks which judgments in an experiment must stay with people.
Picture a team a few years into chaos engineering. At first every drill took three to five engineers half a day to prepare; then it was turned into a platform that picks services, injects faults and aborts on degraded metrics, every day, on its own. The next step looks obvious — remove people entirely and let experiments run silently in the pipeline like unit tests.
Allspaw argues this hides a misunderstanding. Chaos engineering exists because in a complex system nobody fully understands the system; everyone's mental model is off somewhere. Experiments are valuable because they let people see the gap between model and reality. If experiments run fully automatically and nobody reads the results, the system may have been "tested", but nobody's understanding has improved. Left unsolved, the team ends up with a system that is ever more automated and ever less understood.
The chapter splits an experiment into three questions and asks who should answer each:
A common success metric is "vulnerabilities found this quarter". Allspaw's view is different: the real output of an experiment is a change in what the participants understand about the system. An experiment where "nothing broke" can be worth more than one that finds a small bug, if it reveals that a fallback path has never actually fired or that the dashboards would not have shown the problem at all.
That yields an actionable test: an experiment write-up must answer "what we expected, what happened, where the gap is", not just list tickets to fix. Without the "expected" column, an experiment degrades into an automated test.
The chapter's theoretical background comes from human factors research: Lisanne Bainbridge's classic 1983 paper "Ironies of Automation" observed that once routine operation is automated, people are left with only the hardest job — handling the abnormal — while their skills decay and their feel for the live situation fades because they no longer operate by hand.
Applied to chaos engineering: if a platform picks targets, runs experiments and closes tickets on its own, engineers lose one of their main windows into the system. When a real incident arrives — precisely the case automation did not cover — people are less familiar than before. So the goal of automation should be to extend human capability (let people see more and try more), not to replace human judgment.
Automatic abort conditions are essential: for example, if the experiment group's success rate falls below the control group's by more than an agreed threshold, stop and roll back at once. But thresholds only cover failures you anticipated. When something outside them appears during the run — an unfamiliar error in the logs, another team reporting odd behaviour — whether to stop or shrink the scope needs someone present to judge.
Conversely, people can become the bottleneck: if every experiment needs a senior engineer watching throughout, experiment frequency is capped by human calendars. The sensible answer is tiering: well-understood, repeatedly run experiments go to the platform with automatic thresholds; new hypotheses, new targets and anything touching the critical path need a person to design, attend and review.
The core trade-off: automation buys frequency and consistency at the cost of human understanding; human involvement buys understanding and context at the cost of scale.
| Step | Give to automation | Keep with people | Cost of getting it wrong |
|---|---|---|---|
| Target (why) | Rank candidates by dependency graph and traffic | Set priority from incident history and team doubts | Fully automatic: you test what is easy, not what is uncertain |
| Timing (when) | Avoid known peaks and release windows | Judge team state and business events | Run while the team is exhausted and the drill becomes a real incident |
| Execution (how) | Inject, split traffic, abort on thresholds | Design hypothesis and control | Fully manual: slow, steps missed, hard to repeat |
| Abort | Quantifiable thresholds (e.g. success-rate gap > agreed value) | Anomalies outside the thresholds | Thresholds only: unanticipated failure modes slip through |
| Interpretation | Generate comparison reports | Answer "expected vs actual" | Nobody reads them: the system was tested, nobody learned |
No row is either/or; the question is where the line goes, and it should move right as an experiment goes from unfamiliar to familiar.
Industry practice broadly bears out the "automate execution, keep judgment human" line. Netflix's later ChAP platform automated experiment execution: it splits a small slice of traffic into control and experiment groups and stops automatically when metrics diverge (see Ch16 on continuous verification); but what it automates is the "how" — which experiments are worth running and what the results mean remain human work. Asked in an interview "should chaos experiments be fully automated?", a good answer is neither yes nor no but naming the three kinds of judgment you cannot hand over.
Industry evidence
① Core claim: the output of chaos engineering is a more accurate human understanding of the system, not a bug list.
② The pain: once a platform exists, it is easy to take people out of the loop — the system is tested, nobody learns.
③ Three questions: why (target) and when (timing) are answered by people; how (execution) can be automated.
④ Every write-up needs an "expected vs actual" column, or it degrades into an automated test.
⑤ Ironies of automation (Bainbridge 1983): the more complete the automation, the rustier people are when something abnormal happens.
⑥ Aborting: thresholds stop the anticipated failures automatically; anomalies outside them need human judgment.
⑦ Tiering: people attend new experiments; familiar regression experiments go to the platform while people step back to approval and spot-checks.
⑧ Interview answer: don't argue about "fully automated or not" — name the three kinds of judgment you can't hand over.