TOPIC 31 · PHASE E

Coevolution and the Red Queen

Running to stay in place

2026-08-17 · Adaptation & Evolution

The word "progress" quietly assumes a frame of reference that holds still. In a large class of systems the frame is assembled out of your opponents — take a step forward and it steps with you. The improvement is real; the payoff can be exactly zero.

In any neighbourhood, the first family to enrol their child in weekend tutoring genuinely gains something. Three years later everyone has enrolled, the rankings are back where they started, and every family is spending six extra hours a week plus the fees. Nobody cheated, nobody blundered, every single step was rational — and the collective result is a higher cost floor with the ordering untouched.

That structure is more common than it looks: antibiotics and bacteria, spam filters and spammers, advertising and attention, defence and offence, even you and your own tolerance to a drug. They all share one mechanism — how well you are doing depends not on how good you are, but on how much better you are than your opponent right now; and your opponent is changing too. That is coevolution.

The adaptation in the previous few topics faced a stationary opponent: Topic 28 climbed a fixed fitness landscape, Topic 29's payoff table was handed to the players. This topic removes that last assumption — the landscape is worn into shape by everybody else's adaptation, and it shifts the moment you move. With that one change, equilibrium goes from being the normal state to being the rare one.

01You Are Not Climbing a Hill; the Hill Is Moving

Start with the word "fitness". It does not mean strong, good, or advanced. It means one thing: in this round, how many more offspring (or units of market share, or surviving machines) you leave than the others do. It is a ratio by construction, never an absolute quantity.

Topic 28 drew it as a terrain map: your strategy or trait on the horizontal axis, the fitness of that position on the vertical, and you walk uphill → ref · NK fitness landscape. That picture carries an inconspicuous assumption — the terrain was drawn once and holds still. As long as it holds still, reaching a summit means permanently owning the best position available.

Coevolution deletes that assumption. Your fitness depends on your opponent's current state, and your opponent is climbing its own slope; every step it takes redraws your terrain. The result is a distinctly uncomfortable situation: your traits have not regressed at all, your absolute capability is still rising, and yet the ground beneath you has dropped — because what changed was the hill, not you.

The same improvement is worth different amounts in the two worlds ① Start: mid-slope ② Fixed terrain ③ Opponent evolves held old terrain climbing = improving summit reached and held same improvement, back mid-slope horizontal = your strategy or trait vertical = what it is worth right now (fitness) In ③ you never stepped back — the height fell because the hill moved, not you
Delete the assumption that the terrain holds still, and "reaching the summit" stops being something you can finish.

That picture is exactly what Lewis Carroll's much-quoted line means. In Through the Looking-Glass (1871) the Red Queen drags Alice along at a sprint, and when they stop Alice finds they have not moved: "Now, here, you see, it takes all the running you can do, to keep in the same place." In 1973 the palaeontologist Leigh Van Valen borrowed the line to name a statistical pattern — across large numbers of fossil taxa, how long a group had already existed barely changed its probability of going extinct in the next interval. Experience had not accumulated into safety. That is the Red Queen hypothesis: every species keeps getting better, and precisely because everyone is getting better, nobody ends up safer.

🌀 Literature & art · why fashion has to keep changing In 1904 the sociologist Georg Simmel explained fashion like this: an elite adopts a marker to set itself apart, those below imitate it, and once the marker is widespread it no longer separates anyone, so the elite must switch. That is this section's mechanism exactly — the "height" is distinctiveness, and it is set by where the imitators stand, so when they move the hill collapses. The non-obvious consequence: the pace of fashion is not set by taste, it is set by the speed of copying. Fast fashion did not adapt to a faster cycle; it manufactured one.

02The Smallest Model: Why Two Equations Go in Circles

In 1926 the Italian mathematician Vito Volterra was handed a puzzle from the fisheries. During the First World War fishing in the Adriatic all but stopped, so every kind of fish should have become more plentiful; yet the statistics compiled by his son-in-law, the biologist Umberto D'Ancona, showed that the share of sharks and other predatory fish in the catch had gone up. Why should fishing less benefit the eaters rather than the eaten?

He wrote down two lines (Alfred Lotka in the United States had reached the same equations a year earlier, coming from chemical reactions). They are now called the Lotka-Volterra equations → ref · Lotka-Volterra model:

dx/dt = αx − βxy   dy/dt = δxy − γy x is the number of prey (say snowshoe hares), y the number of predators (say lynx). α is how fast hares breed, γ how fast lynx starve, and β and δ set how often the two run into each other — the two counts are multiplied, because they have to meet for anything to happen.

Four parameters, two variables, no randomness, no weather, no seasons. Run it and it starts going in circles all by itself: hares increase → lynx increase after them → hares get eaten down → lynx starve → lynx decline → hares recover.

Four parameters, zero coincidences — the cycle is forced by the lag ① In time: the predator peak lags ② Phase plane: loops, never rest prey (hare) predator (lynx) each one's long-run mean lag time → The lynx peak trails the hare peak by ~1/4 cycle fixed point = long-run mean prey → predators Three starts; none spirals in, none flies out
Time series on the left, the same run plotted on the prey-predator plane on the right. What drives the cycle is not an external cause — it is the lag.

Two results are worth keeping. The first is the lag: the predator peak always follows the prey peak by roughly a quarter of a cycle, which means the current predator count is a report on past prey. It is chasing a target that has already left. The second is stranger. Work out the long-run averages and the mean number of prey depends only on the predator's parameters (it equals γ/δ) — it has nothing to do with how fast the hares breed. If you want more hares on average, change something about the lynx; changing the hares does nothing.

That is how the fisheries puzzle resolves. Apply an indiscriminate kill to both sides (nets that take small fish and sharks alike) and the mean prey number falls while the mean predator number rises; stop that indiscriminate kill and the prey side is what gains — hence the wartime rise in the shark share. The result is now called Volterra's principle: an intervention applied without discrimination has opposite effects on parties at different levels of the chain.

🎯 THE DECISION LINE

When you see a rival's scale peak, do not treat it as news. In a lag-coupled system it is most likely the echo of your own expansion one round ago. A concrete test: correlate the rival's scale metric against your own metric at the same time and against your metric a quarter-cycle earlier. If the lagged correlation is clearly higher, you are looking at an echo — and reacting to it means firing at something that no longer exists.

🌀 Agriculture & ecological engineering · the pests that spraying multiplies After broad-spectrum insecticide is applied at scale, certain pests come back worse than before — the brown planthopper in rice is the classic case. The reason is Volterra's principle: the spray is indiscriminate, killing both the pest and the things that eat the pest, and an indiscriminate kill structurally favours the party lower down the chain. The practical and counterintuitive consequence: failed pest control may mean the spray was too broad, not too weak; what needs replacing is the target, not the potency.

03Why Sex Exists: the Red Queen's Best Evidence

Biology has an embarrassingly blunt problem: sexual reproduction is a terrible deal. An asexual population that produces only daughters reproduces about twice as fast, because males do not bear young — half the productive capacity is spent maintaining individuals who do not produce. On arithmetic alone, asexual lineages should have swept the board long ago. They did not.

The Red Queen's answer is that diversity in your offspring is not a luxury, it is fortification. Parasites (and bacteria, and viruses) reproduce far faster than their hosts, and they track whichever host genotype is commonest, cutting a key to fit it. Copy yourself exactly and you are the lock that key already opens. Sex reshuffles the genome every generation, so the lock you hand over is a new one.

The mechanism has a name: negative frequency-dependent selection — a type's advantage falls as it becomes common and returns as it becomes rare. Note how this differs from a vague "diversity is good": here the causal claim is sharp and measurable — a type's fitness is a decreasing function of its own frequency one step earlier.

Being ahead is itself the reason you get tracked share of host genotype A share of parasites specialised on A parasites catch up A is rare = safest right now time → No type stays ahead; the advantage comes from being uncommon, not from being better
Schematic. Real data is never this tidy, but the phase relation is measurable: the parasite infection profile lags the host's prevailing type.

This is not just paper reasoning; two bodies of evidence carry it. One comes from a New Zealand freshwater snail (Potamopyrgus antipodarum) in which sexual and asexual individuals coexist. Curtis Lively found in 1987 that the heavier the trematode parasite load in a body of water, the higher the proportion of sexual individuals; where parasite pressure is low, asexual clones nearly monopolise — exactly what the twofold advantage predicts. The other is a causal test in the lab: in 2011 Morran and colleagues coevolved nematodes with a bacterial pathogen and found that when the pathogen was allowed to evolve alongside, the worms shifted to outcrossing, while populations forced to self-fertilise were driven extinct. Freeze the pathogen so it cannot evolve, and the effect disappears.

🎯 THE DECISION LINE

Anywhere an opponent is watching you, standardising everything onto the single currently-best option is slow suicide — it hands the whole system over as one lock. Concretely: in critical positions (suppliers, crop varieties, mirrored systems, authentication methods) keep at least one option that is worse but different, and book its cost as an insurance premium rather than as waste. The warning sign is easy to spot: the moment you can say "we use X for everything," you are the commonest genotype. Bananas are the standing lesson — a fungus erased the previous dominant cultivar, and its replacement, the Cavendish, is again a single clone, so the same play is running again.

🌀 Economics & institutions · what happens after a strategy is published McLean and Pontiff examined 97 variables that academic papers had shown to predict stock returns. After publication, the portfolios' returns fell by an average of about 58%, and their correlations with each other rose. That is negative frequency dependence in a market. The actionable shift is in the question you ask: stop asking "does the logic still hold?" and start asking "how much money is currently running it?" — the decay tracks crowding, not correctness.

04Two Kinds of Coevolution, Not to Be Confused

The snails and their parasites never got more "powerful"; they only took turns being the common type. But other coevolution looks nothing like that — both sides raise the stakes step by step and never go back. These are two regimes, and applying the wrong one leads to opposite decisions.

The escalating kind is an arms race. The rough-skinned newt of western North America carries tetrodotoxin (TTX, the pufferfish poison) in quantities that could kill a roomful of people, and the local garter snakes eat them anyway, thanks to a handful of amino-acid changes in a sodium channel. Where the newts are more toxic, the snakes are more resistant, each side ratcheting the other upward. The signature of this regime: the trait has a direction, the cost rises monotonically, and retreat is not available.

The other kind is fluctuating selection — the previous section's picture, where frequencies oscillate with no net direction, and an old model can win again a few generations later. Coevolution experiments with bacteria and their phages have produced both regimes, depending on conditions.

Watch the vertical axis: one climbs, the other only circles ① Arms race ② Fluctuating selection prey toxicity predator resistance gap unchanged absolute trait value → share of each type → time time cost rises monotonically, no way back no direction; old models win again The test is not "how fierce is the competition" but "after two years off, how long to get back to today"
Stop in the left regime and you fall behind permanently; stop in the right one and you have merely changed position. Knowing which side you are on matters more than knowing how fast you are running.

One more thing that often gets skipped: coevolution has a geography. John Thompson's geographic mosaic theory holds that the same pair of species is coupled with wildly different intensity in different places — some sites are hot spots where the two are locked together, others are cold spots where they barely interact. Broad sampling of the newts and snakes bears this out: in many populations toxicity and resistance simply do not match, and snakes occasionally leap, via a single decisive mutation, far above the local toxin level and escape the race entirely (while the prey side is never observed to get ahead). So "we are in fierce competition" is a claim that first has to say where.

🎯 THE DECISION LINE

Classify every competition you are funding, using one question: "If we stopped this for two years, how long would it take to get back to where we are today?" If a few weeks, it is fluctuating — you can start and stop with the rhythm. If more than a year, it is an arms race: it will only get more expensive, and winning it does not buy safety. For that kind, what ends it is never running faster. It is moving to a dimension the coupling does not cover, or a mutual limit both sides can verify.

🌀 Medicine · the flu vaccine is reformulated yearly, the measles one is not The usual explanation is that flu is "craftier", but the mechanism is not craft: influenza's surface proteins tolerate a great many changes that cost it nothing functionally, so under immune tracking it can keep drifting; measles has its critical sites locked down by function, so decades on there is still essentially one serotype. That yields a usable test: what forces annual reformulation is not how dangerous a pathogen is, but how many dimensions it is free to vary in. The same question transfers to any detection-versus-evasion contest — start by counting the dimensions your opponent can move in for free.

05Where This Breaks Down

"Red Queen" is one of the most useful and most abused phrases in complexity science — it slides very easily into being a decorative way of saying "competition is fierce, so keep innovating," which is empty. Here are its four real boundaries.

First, Van Valen's law of constant extinction is not uniformly supported by later data. Palaeontology now runs two explanations side by side: the Red Queen (organisms pressing on each other) and the Court Jester (climate, tectonics, and other indifferent external events). Michael Benton's 2009 review concluded that they operate at different scales in space and time: biotic interaction dominates locally and over short spans, while climatic and geological events dominate regionally and over millions of years. Explaining a mass extinction with the Red Queen is usually reaching for the wrong level.

Second, Lotka-Volterra's "eternal cycle" is an artefact. Those closed orbits are structurally unstable: add one realistic touch — a ceiling on the hares' food supply — and the neutral loops immediately collapse into a spiral converging on the fixed point, or into a limit cycle of fixed amplitude. Do not read the loops as a prediction. The textbook hare-lynx cycle is not evidence for them either: Charles Krebs's team ran an eight-year field manipulation in the Yukon and found that excluding predators doubled hare density, adding food tripled it, and doing both raised it elevenfold — far more than additive, which says the cycle involves at least three trophic levels and does not fit inside two equations.

Third, not all coevolution is a fight. Mutualism is coevolution too — figs and fig wasps, legumes and rhizobia — and its direction is not escalation but deepening mutual dependence. Treating coevolution as a synonym for arms race is where the concept is most often misapplied. The test is the same one: look at whether each side's cost curve is climbing or falling.

Fourth, and most important: it has to be falsifiable. What the Red Queen actually claims is narrow — absolute improvement fails to convert into relative position, an advantage decays as it spreads, costs rise monotonically. All three are measurable. "Competition makes everyone better" excludes nothing; any outcome can be fitted into it. A concept that answers every counterexample with one more auxiliary clause ("that region was Court Jester", "that site is a cold spot") while issuing no new predictions has stopped being a tool and become rhetoric.

🎯 THE DECISION LINE

Before saying "this is a Red Queen situation", clear three bars: ① is your fitness genuinely a function of your opponent's current state rather than of a fixed external environment (if you are not sure, go look — did your results change during the year the opponent sat still?); ② is there a measurable quantity in motion — a frequency, a cost, a relative gap, any one of them; ③ what new prediction does the claim make, and what result would count as refuting it. If you cannot answer the third, delete the phrase and describe your mechanism in plain words.

🌀 Western philosophy of science · Lakatos on degenerating programmes Imre Lakatos distinguished progressive from degenerating research programmes, and the difference is not whether counterexamples exist — they always do — but whether the programme grows by making new predictions or survives by patching counterexamples away. That is precisely the ruler the fourth point needs, and it yields a hard rule for every concept on this site: judge whether it is still worth using not by how much it explains, but by what its most recent novel prediction was. Explanatory power can grow without limit, and that is exactly the bad news.

🎒 Scenarios · BigCat

  1. Health & energyThe recurring situation: something "doesn't work as well as it used to," so you raise the dose — a third coffee, two painkillers instead of one, another notch of training load. Two very different couplings are mixed together here and are worth separating. Receptor tolerance (caffeine, analgesics) is reversible: stop for a couple of weeks and the baseline washes back. Resistance (a microbial community shaped by repeated antibiotic courses) is not: stopping does not send it back. What to change: replace "did it work this time?" with a record of the slope of the dose required to get an effect over time, and tag each item as reversible or not — only the reversible kind can be solved by taking a break, and the irreversible kind has to change dimension now (sleep, load scheduling, narrow-spectrum as the default).
  2. Leading teams & organisationsThe recurring situation: quarterly planning goes down the competitor's feature list item by item, every item gets shipped, and a year later the relative position is unmoved while the surface you must maintain has permanently grown. What to change: add a column to roadmap review, replacing "does the competitor have it?" with "after this ships, has our relative position changed? What does it cost per year to keep?" What to stop: delete every item whose only justification is "because they shipped it." Use the test above — if the capability could be paused for two years and rebuilt in a few weeks, it does not belong in this quarter.
  3. Writing & this learning site itselfThe recurring situation: one issue's structure or diagram style lands well, so the next few issues quietly reuse it. Negative frequency dependence eats it: the more fluent a form becomes, the lower the reader's marginal response, while your own fluency makes reaching for it again more tempting. What to change: stop reading each issue's response in absolute terms and start recording the response to the nth use of a form as a ratio to its first use; when that ratio falls twice in a row, force a change of form — even though the first attempt at the new one will be rougher.

🌀 Crossings

🧠 Going Deeper

If equilibrium is rare, why do so many industries hold the same shape for years?

Be careful not to read "the outcome is unchanged" as "nobody is running." The Red Queen describes exactly this case: stable positions with continually rising inputs. So the test is not in the outcome but in the cost. Plot relative share and absolute investment on one chart over several years: share flat, investment climbing, is a textbook Red Queen, not a steady state. A real steady state has both lines flat.

Isn't "changing dimension" just starting another arms race?

Often it is. The meaningful difference lies in what the new dimension's coupling strength and cost slope look like: how fast the opponent can follow, and what following costs them. If both match the old dimension, you have only changed the venue. A genuinely valuable move finds a dimension where the opponent's adaptation is hard-constrained — the way the measles virus cannot move its critical sites.

If sex is for defeating parasites, shouldn't asexuality return wherever parasite pressure is low?

It does happen, and that is one of the Red Queen's falsifiable predictions — the New Zealand snail data broadly fits. But other explanations for sex are competing for the same job: purging the accumulation of harmful mutations (Muller's ratchet), and recombining beneficial mutations into one lineage. These are not mutually exclusive and probably stack. It is also a good illustration of what makes an explanation worth having: one that predicts a geographic difference is worth far more than "diversity is good."

Is the Red Queen the same thing as a rat race?

Not quite. The Red Queen is descriptive: absolute improvement does not convert into relative position. "Rat race" usually carries two extra claims — that participants cannot exit, and that total output has not increased. The second is often false: arms races frequently do produce real technology (radar, antibiotics, the dreadnought's propulsion). Keeping them apart clarifies a great deal: mechanism is one thing, value judgement is another.

📚 Further Reading