The sharpest cuts come from inside
2026-08-30 · Methods and Limits
The louder a field is praised from outside, the more likely it is having an unglamorous fight on the inside. The best critical literature on complexity science was written by its own members — and the lines outsiders love to quote are mostly the ones the insiders buried long ago.
You have heard sentences like these: "it's a complex system, linear thinking won't work here", "our organisation needs more emergence", "we're operating at the edge of chaos". They land, because each sounds like it is saying something. But ask one follow-up — what does it predict? what observation would make you take it back? — and there is usually nothing behind it.
Here is the twist: the fiercest critics of that habit are not humanities scholars, nor irritated physicists. They are the field's own core members. Melanie Mitchell, who wrote the best popular book on complexity, personally overturned the most famous experimental result behind "the edge of chaos". Aaron Clauset built a statistical test and then used it to audit his colleagues' published power laws, failing a good many. Criticism from outside mostly misses; criticism from inside mostly lands.
The previous topic asked how a single model goes wrong. This one asks the bigger question: what is the field as a whole accused of, and which charges stick. Four charges, judged one at a time — and then a final section for something just as important, because the critique has a boundary of its own. "It's complex" used to end a discussion and "it's complex" used to explain everything are the same move.
First, a test that the next four sections all lean on: the value of a claim lies not in how many phenomena it fits, but in what it rules out.
"On a two-dimensional square lattice where each site is occupied independently with probability p, no cluster spans the lattice when p is below 0.5927" is a hard claim, because it forbids an enormous number of observations. Run the experiment at p = 0.5, see a spanning cluster, and the claim is dead.
"This organisation is at the edge of chaos" fits any organisation. No observation would make the speaker withdraw it. It rules nothing out, so it bears no load in an argument — it is an exclamation wearing the clothes of a statement.
The "edge of chaos" deserves its own account, because it is the cleanest specimen of this charge. Around 1990 Chris Langton, studying cellular automata (the grid-and-simple-rules models of Topic 4 → ref · Cellular Automata), proposed that between "rules too rigid" and "rules too random" lies a narrow band where systems compute best. In 1988 Norman Packard had reported something even more seductive: when a genetic algorithm (a search method that mimics evolution → ref · Genetic Algorithms) evolves cellular automata to perform a computational task, the evolved rules cluster by themselves near that critical band.
That result is too good not to repeat: evolution moves towards the edge of chaos. It went straight into countless books on management and innovation.
Then, in 1993, Melanie Mitchell, Peter Hraber and James Crutchfield redid the experiment. The clustering did not reproduce. Their conclusion was that what Packard saw was an artifact of the genetic algorithm's own dynamics together with that particular fitness function — not evidence of systems evolving towards criticality.
That was over thirty years ago. "Keep your organisation at the edge of chaos" is doing fine — precisely because it detached from the falsifiable claim underneath and became pure metaphor. Charge one: upheld.
Whenever a complexity term shows up in a conclusion, ask one fixed question: "what does it forbid?" — i.e. what observation would make the speaker admit they were wrong. If there is no answer, strike the term from the conclusion. It may stay in the question-asking phase as a lead; it cannot serve as a reason. The check takes ten seconds and stops most of them.
The second charge is the common one: the field always has a mechanism for what already happened and little to say about what hasn't. In 1995 the science journalist John Horgan published a famous broadside in Scientific American titled "From Complexity to Perplexity", in which even researchers at the Santa Fe Institute doubted the promised unified theory would arrive; he later coined "chaoplexity" for chaos and complexity together in his 1996 book.
But the charge smuggles in an undeclared premise: prediction means naming the date and the size. There are in fact three kinds of prediction, and they demand entirely different evidence.
Topics 8 and 11 already argued the forfeit: chaos means errors grow exponentially, and computational irreducibility means some systems have no shortcut but running them at full length. So "can't tell you when and how big" is not a failure to deliver — it is an assertion that it cannot be done. Scoring the field on it is like judging a swimmer by their sprint time.
That defence is only half the job, though. The other half: were the remaining two kinds delivered? An honest scorecard —
Delivered: percolation thresholds are computable for specific lattices and networks, and they hold up when you move to another system → ref · Percolation; the basic reproduction number R₀ crossing 1 flips an outbreak from dying out to spreading, and that threshold is the skeleton of epidemic policy → ref · The SIR Model; and critical slowing down as an early-warning signal was tested in a genuine field experiment in 2011, when researchers gradually stocked predatory bass into one lake to push it over a food-web transition while an adjacent lake served as an untouched control. More than a year before the transition completed, variance and autocorrelation did rise in the manipulated lake and not in the control. That was designed and recorded in advance.
Not delivered: earthquakes. Self-organized criticality (SOC) in fact implies that magnitudes are unpredictable, which the field admits; in 1997 a group of seismologists argued in Science that earthquakes cannot be predicted in any practical sense, and no repeatable forecast has overturned that in the decades since. As for the 2008 financial crisis — complexity economics offers an elegant account after the fact (leverage, network coupling, homogenisation), and essentially no archived forecast before it.
Charge two: partly upheld. The part that sticks is not "it can't do point prediction". It is that practitioners routinely let themselves stop distinguishing "I can explain this" from "I predicted this" — a mechanism write-up composed afterwards and a forecast sealed beforehand look identical on the page. Only a timestamp separates them.
Take any conclusion produced by complexity methods and classify it first: point, distributional, or threshold? Then look for a timestamp. The three have different acceptance tests — a distributional claim is judged by long-run frequencies, a threshold claim by whether the number is computable and reusable in another system. Apply the same to your own judgements: write them down with a date, or six months later you will remember having "seen it coming".
The third charge is technical, and the most bluntly judged: this field sees power laws everywhere, and a good share of them are not power laws.
The cause is almost optical. On log-log axes the eye readily reads a gentle curve as a straight line, and lognormal distributions or exponentially truncated ones look much the same when plotted. In 2009 Aaron Clauset, Cosma Shalizi and Mark Newman supplied a uniform procedure — maximum-likelihood estimation of the exponent, a goodness-of-fit test, then a head-to-head comparison against alternative distributions — and ran a batch of widely cited datasets through it. Many failed → ref · Identifying and Misreading Power Laws. In 2019 Anna Broido and Clauset pushed the method across nearly a thousand real networks and concluded that strictly scale-free structure is empirically rare; for most of them a lognormal fits as well or better → ref · Preferential Attachment.
Note the direction of travel. Both papers were written by insiders, using the field's own data. The charge is upheld — but it was raised and enforced from within, which is evidence of health rather than sickness. A field is in real trouble when nobody does this, not when somebody does.
"Everything is a network" is subtler, because it does not look like a claim at all. Drawing a pile of entities as a network feels like discovering structure. It is a modelling choice: you chose which relation counts as an edge, over what time window, and how weak a tie you discard. Change those three and you change the graph.
So a sentence of the form "X is a hub in this network" has to travel with its construction rules. Drop the rules and readers will think you are describing the people. You are describing your spreadsheet. Charge three: the power-law half is upheld and being repaired; the "everything is a network" half is partly upheld — networks are a fine tool, and the failure is treating a representation as a discovery.
Before reporting any network result, state three things: the edge definition, the time window, and the threshold below which ties were dropped. Then run a substitution check — rebuild the graph under an equally defensible edge definition. If the hub changes, your conclusion belongs to your construction, not to the system, and must be downgraded in the write-up. Likewise, run a goodness-of-fit test before saying "power law"; a line that looks straight on log-log axes is not evidence.
Put the first three charges together and a question follows: other fields exaggerate and misuse too, so why does complexity science look so much worse? The answer is structural rather than moral — it has four exposures others do not.
First, its method is analogy. The field's whole claim is that one mechanism recurs across unrelated domains: sandpiles, extinctions and crashes share a statistic. And a sound transfer and a bogus one are identical at the level of the sentence — both say "the same mechanism is running underneath these two things". The only difference is whether the premises travelled too.
Second, it owns no data. Physics has laboratories, epidemiology has case registries; complexity science has no observational object of its own and always works on somebody else's data. Where that data is dirty — how it was collected, who is missing, when the definitions changed — is known only to people inside that field. The visitor receives a clean table and never sees what was swept under it.
Third, referees understand half a paper. On an interdisciplinary submission, the mathematically fluent referee often cannot see the flaw in the empirics, and the empirically fluent one is intimidated by the mathematics and assumes it is fine. This is nobody's conspiracy; it is structure: no one is equipped to say no to the whole paper.
Fourth, naming has a market. In a January 2000 newspaper interview Stephen Hawking said "I think the next century will be the century of complexity" — a line quoted endlessly since. It is an expectation voiced in an interview, not a research finding, yet it travels with more weight than any paper. Once a field owns a sentence like that, some fraction of the people who arrive have come for the sentence.
Before moving a mechanism into a new domain, write out the premise list it needs (SOC, for instance, needs slow accumulation, threshold release and local coupling), check the new domain item by item, and state which items fail. A failure does not make the connection worthless; it means the connection is downgraded to an analogy and may not enter the conclusion. Do this and most "cross-disciplinary insights" evaporate on the spot. The survivors are the ones worth having.
Now turn the gun around. The critique overreaches too, and in exactly the way it accuses its target of overreaching.
"No unified theory, therefore not a science" — rejected. It sounds forceful and uses the wrong standard. Ecology has no unified theory. Neither does geology, nor immunology, and nobody concludes they are not sciences. The test was never "is there a Theory with a capital T" but "are there specific models that can be checked and overturned". Of those there are plenty: percolation, SIR, Kuramoto, NK landscapes, preferential attachment — each makes definite assertions, and each has been corrected (the systematic rebuttal of scale-free networks in Topic 22 is a model being disciplined by data, live).
"It failed to predict 2008" — wrong scorecard. Section 2 took this apart: judging a field by point prediction when it explicitly forfeits point prediction is using the wrong instrument. The question is whether its threshold and distributional claims were tested. Note that this defence cannot be flipped into an amnesty: the price of forfeiting point prediction is stricter testing on the other two kinds, not exemption from testing.
And the one that matters most: the critique itself can become a zero-cost posture. "It's complex." "Systems are unpredictable." "Linear thinking doesn't apply here." These forbid nothing either. Using "complexity" to end a discussion and using it to explain everything are the same move in two directions. The first sounds more modest, which is why it is harder to catch.
By the same token, Sokal-style reckonings have an overreach of their own: going from "some humanities scholars misused physics terms" to "cross-disciplinary transfer is worthless" also drops the domain of validity. What that episode demonstrated was a failure at one refereeing stage, not the bankruptcy of analogy.
So the honest boundary sits roughly here. What this field delivers best is the shape of mechanisms — where the threshold is, what shape the distribution has, how structure drives dynamics. What it delivers worst is anything of the form "when, how big, who". Wherever it starts sounding most like a story, that is where you should insist on hearing what it forbids.
Audit your own vocabulary. List the complexity words you have used in the past three months — emergence, criticality, resilience, antifragility, network effects, positive feedback — and write one line for each: what does it forbid? The ones you cannot answer for do not have to be retired, but they must be downgraded from "a reason in a conclusion" to "a lead when asking questions". Downgraded usage looks like this: not "we're at a tipping point", but "if this really were critical, the accumulation should be rising and recovery times lengthening — go and check those two numbers".
The test governs conclusions, not questions. Plenty of important ideas could not say what they forbade at the outset — continental drift, plate tectonics, the material basis of the gene. The difference is that they later found a checkable form rather than staying suggestive forever. So the right use is to keep a ledger for each vague idea: it currently sits at the "lead" level; when does it move up to "reason"? Three years without moving usually means it never will.
Because criticism costs far more to transmit than a slogan. "Evolution moves towards the edge of chaos" is a thirty-second line in a talk; "that clustering was an artifact of the genetic algorithm and the fitness function" requires the listener to already hold the first sentence. The transmission mechanism is itself a filter: it preferentially passes sentences that need no background to retell — and needing no background is often precisely what happens once the premises have been stripped off.
Because it shifts the burden of proof. Whoever asserts "A caused B" has to make a case; whoever says "it's complicated" appears to be asserting nothing and so appears to owe nothing — while in fact asserting something strong: that your causal account is wrong. The way to test it is direct: ask what they would expect to see if the thing turned out not to be complex. No answer means the sentence was a refusal, not a claim.
There is no fixed answer, but there is a workable substitute question: has it produced a public record of insiders overturning insiders? Plate tectonics took roughly half a century, and that record fills it. Complexity science can in fact hand in that homework — the 1993 replication, the 2009 test, the 2019 sweep — and those say far more about its condition than any pronouncement that it is "the science of the 21st century".