The kind of system that reads your model
2026-08-12 · Adaptation & Evolution
Everything in the first twenty-five issues had one convenient property: sand does not read your paper. From this issue on, the thing being studied reads it — and then changes.
One junction jams solid every afternoon at half past five, so you widen the road by a lane. The engineering is sound; capacity really does go up. Come back six months later: the same jam, only now there are cars on it that never used to come this way.
What failed was not the engineering. It was the phrase "all else being equal." Drivers reconsider which route to take, when to leave, whether to drive at all — and those decisions were the inputs to the traffic chart you started from. You changed more than the road; you changed the premises the chart rested on.
The systems in earlier issues shared one feature: their parts have no idea what they are participating in. A grain of sand does not steepen its angle of repose because the last avalanche was too big, and a transmission line does not learn from a bad trip. The systems from here on are different: their parts carry a model of the system around with them and revise that model according to results. That single difference is enough to make some of the machinery of the previous twenty-five issues stop working. This issue draws the line and says what is available on each side of it.
Start with the vocabulary. Anything that makes its own decisions, and whose decisions affect others, is an agent: a driver, a firm, an ant, an immune cell, a trading program. A system made of many agents that revise their own behavioural rules in response to results is a complex adaptive system, universally abbreviated CAS — the abbreviation is used below.
The term is John Holland's (1929–2015), a computer scientist at Michigan and one of the earliest people at the Santa Fe Institute → ref · Santa Fe Institute. He spent a career on one question: how does adaptation actually happen, and can it be written as a program you can run?
The point is not that agents are clever. Ants are not clever, and immune cells much less so. The point is that two loops are running, orders of magnitude apart in speed:
The fast loop is acting under the current rules — picking a route, quoting a price, striking. The slow loop is the rules themselves being replaced: the ones that did well are kept, copied, imitated by more agents; the ones that did badly are dropped. Physical systems have only the fast loop. Their rules hang permanently outside the system and never join the game. Adaptive systems have pulled the rules into the game.
So "this system adapts" is not a flourish; it is a checkable claim, and you check it by asking whether the slow loop exists: where do candidate rules come from, how do results get back to the rules, and do the losing rules actually disappear? If any of the three has no answer, the loop is broken and the system does not adapt on the timescale you care about — even if it is made of people. A bureaucracy welded shut by procedure, where a trial takes six months to approve and nothing is ever retired for failing, is closer to a sandpile than to an ant colony on a six-month horizon.
Classify the system before acting, using those three questions. No answers → treat it as physical; the mechanisms of earlier issues (thresholds, cascades, power laws) apply directly and publishing your model costs nothing. Answers → every rule, metric or public analysis you release comes with a period before it is adapted around, and the length of that period is estimable (section 4 gives the estimate).
Holland's contribution was not the claim that systems adapt. It was breaking adaptation into three parts, none of which can be missing — broken down far enough to be written as code. The genetic algorithm of his 1975 book is the minimal implementation of exactly those three parts → ref · genetic algorithms.
One: variety. You need several different candidate rules on hand at once. A system with one rule has nothing to switch to and will never adapt however good its feedback is; it can only repeat.
Two: credit assignment. You need some way of attributing a good outcome to a particular rule. This is the hard part, because reward usually arrives late: the game is won on move 200 — does the credit belong to move 37? Holland designed a scheme he called the bucket brigade for this, in which each link passes a little of its reward back to the link before it, round after round, until the early setup moves — the ones whose value is invisible at the time — finally get scored.
Three: replication and recombination. High-scoring rules must be copied more often, and pieces of two rules must be able to combine into a new candidate, so the search is not just local tinkering.
Laying out the three parts buys you something immediately: "this organisation doesn't learn" stops being a complaint and becomes a locatable fault. Is there no variety (everyone knows one way of doing it)? Is credit assignment broken (nobody sees the outcome, or good and bad both get attributed to the same person)? Or is retirement broken (failed practices are never stopped, only supplemented by new rules)? The three repairs have nothing in common, and blurring them together yields advice like "we need a learning mindset," which nobody can act on.
To make a system learn faster, measure one number first: the median number of days between making a decision and being able to see its result. That number is the ceiling on the slow loop's speed. If it is 90 days, stop expecting anything to be calibrated within a quarter. There are only two ways to shrink it — find an earlier proxy signal that correlates with the final outcome, or cut decisions small enough to resolve sooner. Exhortation does not move this number.
Put the first two sections together and you get the most practical result in this issue.
You observe a system made of agents, find a stable regularity, write it into a model, and publish it — "we will assess people on lines of code," say, or "inflation rises once unemployment drops below 4%." At that moment the model becomes a new reward signal for the agents. The machinery of section 2 starts up: among the candidate rules, those that push the signal score well, and high scorers get copied. A few rounds later the regularity is gone.
This is Goodhart's law. Charles Goodhart's 1975 formulation was: any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. The famous one-liner — "when a measure becomes a target, it ceases to be a good measure" — is anthropologist Marilyn Strathern's 1997 rephrasing. It is catchier, and it drops the crucial words: pressure placed upon it. The regularity does not rot on its own; it is crushed by being leaned on.
What matters most is that nobody has to cheat. Economics of the same decade has a colder version: in 1976 Robert Lucas pointed out that a policy model fitted to historical data has parameters encoding how people formed expectations under the old policy; change the policy and expectations change with it, so the parameters were never constants. That is the Lucas critique. The Phillips curve — the stable negative relationship between unemployment and inflation found in 1958 — collapsing in the 1970s is the ready-made example. Nobody lied. Firms and workers simply, honestly, priced in the fact that the government would tolerate inflation. Honest adaptation is enough to destroy a regularity.
Once the mechanism is stated you can also state when it does not happen. A metric gets optimised away because there is a seam between it and the real goal, and because agents can crawl into that seam. So Goodhart fails in three cases: the metric is the goal rather than a proxy, so there is no seam; the agents have no adjustable space; or crawling into the seam costs more than it pays. Those three are also three design strategies — the next section puts numbers on them.
Whenever you publish a metric, write down two things alongside it: one concrete route to raising it that does nothing for the real goal (if you cannot write one, you have not yet worked out whether it is a proxy or the goal itself), and a review date. Add an observation-only quantity that carries no consequences, ideally collected from a source unconnected to the people being assessed. Any metric assessed for more than four consecutive quarters without review should be assumed to have drifted.
"Complex adaptive system" is the most all-purpose phrase in this field. Three boundaries have to be on the table before using it.
First, adapting is not improving. The machinery in section 2 does one thing: keep the rules whose payoff is high right now. Payoff is defined by the environment, and in these systems the environment is mostly other agents — so it can perfectly well push everyone somewhere worse together (every student takes another year of tutoring, every firm adds another round of overtime; the ranking is unchanged and the costs have doubled). "This is a complex adaptive system" therefore predicts no direction at all. Having said it, you know nothing about whether the outcome will be good. It tells you the rules will change, not where they will go.
Second, not everything being modelled adapts — and this can be estimated. Open up the slow loop and the rate at which you get adapted around depends on two measurable quantities: how long one lap takes (make a change, see the result, change again) and how large the adjustable space is (how much the other side can alter). Together they give a rough shelf life. Influenza antigens drift fast and have room to vary, so the vaccine is reformulated every year; measles antigens are stable with almost no usable drift, so the same vaccine has worked for decades. To the same question — will the virus escape? — the answer lies not in the vaccine but in the other party.
Third, the phrase itself slides easily into vacuity. Saying "the economy / the organisation / the brain is a complex adaptive system" has no content unless you can follow it with the candidate rules, the credit-assignment signal, and the retirement mechanism. Without those it excludes no possibility and cannot be falsified. There is a subtler problem too: physics has universality classes (Topic 16), where wildly different systems really do share critical exponents, so conclusions travel. Adaptive systems have no such thing. A result about ant colonies does not transfer to organisations, because behaviour is set by the concrete form of those three components, and the forms are entirely different on the two sides.
Before any plan that depends on staying a step ahead, estimate one ratio: how long the other side takes to complete a try-observe-revise lap ÷ how long you take to update your rules. Above 1, a plan based on information advantage is viable — but write down its expected lifetime and exit condition. Below 1, don't: switch to plans that survive being seen. Align the incentive (so that optimising your metric really does move the goal) or change the structure (so the shortcut does not exist). Both keep working after publication.
Two things. Results about structure rather than parameters: as long as the slow loop runs, certain shapes recur (metrics get optimised away, arms races, lock-in on local optima) — and knowing them does not let you escape them, so publishing does not destroy them. And predictions about quantities with no adjustable space. What genuinely expires is the parameter-shaped result: "under the current rules, X correlates with Y."
Yes, and it has been done often: kill variety (only one approved way), kill credit assignment (results are not published, or good and bad both land on the same person), kill retirement (nothing failing is ever stopped, only supplemented). Each will make a system "stable" on the timescale you care about. The price is that it has no response left when the environment shifts — the stability was bought by spending the adaptive capacity in advance.
It need not. An internal model in Holland's sense can be as thin as an ant's: a rule saying "follow the stronger pheromone trail" already encodes a prediction about the world. All that is required is that the rule change in response to results — not that the agent know it is predicting anything. What rationality assumptions really constrain is standard economic modelling, not this.
No. An organisation without metrics does not become more honest; it hands evaluation to something less visible (impressions, seniority, who talks loudest). This issue points elsewhere: accept that metrics have a shelf life and give them a review date, pair them with an observation-only counterpart, and prefer metrics with a narrow seam — ones where the only route to raising them is doing the work.
Section 4 gives one concrete answer: physics has universality classes, where critical exponents really are shared across unrelated systems, so conclusions travel. Adaptive systems have no equivalent; behaviour is set by the concrete form of the three components. What travels is the vocabulary, not the result. The cheapest guard against metaphor is to name all three components every time — if you cannot, the sentence has no content yet.