Crowds are wise only under conditions
2026-08-16 · Adaptation & Evolution
The same group of people can guess an ox's weight to within 0.75% — and can also be wrong together, in perfect unison, for two thousand years. What separates the two isn't morality or intelligence. It's two or three parameters you can write down.
"Two heads are better than one" and "the madness of crowds" are two proverbs that flatly contradict each other, and the awkward part is that each has a mountain of examples behind it. A crowd at a country fair guesses the weight of an ox and their median lands within one percent of the truth. The same people, pooled together, panic-buy salt, chase a rally, and pass on a rumour with an efficiency that is painful to watch.
The usual way of reconciling this is "it depends" — sometimes crowds are smart, sometimes they're stupid. That says nothing. The real situation is far more specific: smart and stupid are two outputs of the same machine under two settings of its parameters. There is one machine, and there are only two or three parameters. Set them right and a room of ordinary people beats the sharpest person in it. Change one and that same room will lock in an error with remarkable efficiency — with everybody behaving rationally and nobody being foolish.
So "should we trust the crowd?" is the wrong question from the start. The question is: this time, what are those two or three parameters set to?
In 1906, at a livestock fair in Plymouth, England, there was a prize competition: guess the weight of a fat ox after it had been slaughtered and dressed. Butchers and farmers entered, and so did passers-by with no idea what they were looking at. Each wrote a number on a card.
Francis Galton later borrowed those cards and ran the numbers, publishing the result in Nature in 1907: 787 valid entries, median 1207 lb, against an actual dressed weight of 1198 lb. Nine pounds off. 0.75%.
The story has been told to death, usually as "the crowd knows best." But that's a conclusion, not a mechanism — and it's wrong. The crowd did not know best; the overwhelming majority of those 787 people were badly off. What actually happened is that their errors pointed in opposite directions and cancelled.
That sentence can be written as an exact equation — not a metaphor. Let θ be the truth, sᵢ the estimate of person i, and s̄ the arithmetic mean:
The left side is the squared error of the crowd's single answer. The first term on the right is the average of everyone's individual squared errors. The second is the variance among them — how much they disagree. These three quantities form an identity: plug in any numbers you like and it holds. No assumptions, no requirement that anyone be "independent enough." Pure algebra. Scott Page calls it the diversity prediction theorem.
Two consequences follow, and both are far harder-edged than the phrase "wisdom of crowds."
First, the crowd is never worse than the average member. The subtracted term is a variance, hence non-negative. So "people drag each other down" is, under the averaging rule, strictly impossible. Note what this does not say: it says nothing about the crowd beating its best member. That is a different claim, and it often fails.
Second, disagreement is a benefit, not a cost. There are two routes to a smaller crowd error: make each person more accurate (shrink the first term), or make them wrong in more different ways (grow the second). In the equation the two routes have exactly equal standing. Which yields a counterintuitive result: adding a below-average person who is wrong in a completely different way can make the group more accurate. Their own error enters the first term; their disagreement with everyone else enters the second, and the second can be larger.
When deciding whether to pull one more person into an assessment, don't ask "do they know this area?" Ask "will they be wrong in the same place as everyone already in the room?" That is a checkable fact rather than an impression: look at their training, their main information sources, and the direction of their last few misjudgments. If all three overlap with people already present, they add cost without adding accuracy — the diversity term didn't move. Stop doing this: adding reviewers in order of seniority. People of similar seniority were usually taught by the same books and the same experiences, which is a systematic way of shrinking the second term.
🌀 Engineering & the history of technology · Random forests deliberately make each tree worse A random forest trains each decision tree while allowing it to look at only a small random subset of features at each split. Any single tree is clearly worse than one permitted to see everything. Breiman's 2001 generalization bound nails down why: the error is capped by ρ̄(1−s²)/s², where s is the strength of a single tree and ρ̄ is the average correlation between trees — the numerator is asking for decorrelation. Which yields a conclusion that holds for people too and that almost nobody acts on: raising the average quality of members and raising the accuracy of the group are goals that can fight each other. When members are already highly correlated, the right move is to blindfold them differently, not to hand everyone the same additional briefing.
The identity above hides a premise: it assumes each person produces their sᵢ on their own. If the second person writes their number while looking at the first person's, the second term — diversity — collapses on its own.
And the damning part is that this requires nobody to be a follower. A rational person does it too, and has to.
In 1992, Bikhchandani, Hirshleifer and Welch gave the cleanest model of this process, the information cascade. Its laboratory version runs like this.
There are two urns on the table. Urn A holds 2 red balls and 1 blue; urn B holds 2 blue and 1 red. The experimenter picks one at random (50/50) and tells nobody which. Participants come up one at a time, privately draw a ball and look at it — that is their private signal — put it back, and then announce publicly which urn they think it is. Everyone downstream hears all previous announcements but never sees anyone's ball.
The model has three consequences, and each maps onto something real.
Cascades start easily. All it takes is the first two signals happening to agree. The trigger can be pure luck — the urn is B, the first two people happen to draw red, and the whole room goes wrong to the end. So "surely there must be something to it, look how many people say so" carries no weight at all wherever cascades are possible.
Cascades are brittle. Because the ninety-eight people behind carry no information at all, one genuinely new piece of public evidence can flip the entire queue on the spot. Consensus that looks monolithic often falls at a push — its solidity and its fragility are the same fact seen twice.
Nobody did anything wrong. Every participant made the Bayes-optimal choice. This is not conformity, cowardice, or a lack of backbone. Diagnosing it as "poor independent thinking" turns a structural problem into a character problem — and character can't be changed, while structure can. → Information cascades
The cleanest real-world measurement comes from the music experiment Salganik, Dodds and Watts published in Science in 2006. They built a download site stocked with 48 songs by unknown bands and randomly assigned 14,341 participants across 18 mutually invisible parallel "worlds": in 16 of them you could see how many times each song had already been downloaded, and in 2 of them you couldn't — you had to listen and decide.
Two results. The first is unsurprising: in the worlds where download counts were visible, inequality was amplified — winners won bigger. The second is the damaging one: those 16 worlds disagreed with each other. The same song ranked 1st in one world and 40th in another. Quality was not irrelevant (in the independent worlds, the best songs rarely sank and the worst rarely rose), but everything in the middle was essentially arbitrary, decided by the accidental clicks of the earliest arrivals.
Which gives a reading you can apply immediately: use "popularity" as a proxy for quality and what you are actually measuring is how many times over the taste of a few early arrivals got amplified.
Wherever you collect judgments, turn off the live counter: no running vote tally before the vote, no distribution of submitted scores during review, no "the majority view is…" before the discussion. When something must be shown, show reasons, not counts — a reason is new information, a count is not. One more thing you can do today: when gauging how strong a consensus is, count how many mutually independent information sources exist, not how many people hold the view. Where cascades run, those two numbers can differ by two orders of magnitude.
🌀 Philosophy of science · A citation count is not an evidence count In 2009, in the BMJ, Greenberg flattened out the entire citation network behind one medical belief: 242 papers and 675 citations on the claimed relationship between β-amyloid and inclusion body myositis. He found that "authority" grew like this — papers that refuted the claim were systematically under-cited, while a mass of papers containing no data at all propped the belief up by citing each other. This is the academic edition of the urn experiment: every later paper cites its predecessors perfectly reasonably, but from the third one onward no new observation enters. Which yields a directly usable test: "cited 500 times" and "supported by 500 independent datasets" differ by orders of magnitude. To gauge the strength of a claim, walk back along the citations and count how many papers at the bottom actually measured anything.
The first two sections were about inputs: whether people are independent, and whether they are wrong in different ways. This one is about the exit — how those judgments get twisted into a single answer.
In meetings, in engineering, in institutions, this step is almost always treated as a formality: "let's take a vote," "let's average it." It isn't. Give the same set of opinions a different aggregation rule and the answer moves by an order of magnitude.
Concretely. Seven people estimate how many more weeks a project needs: 2, 3, 3, 4, 4, 26, 30.
So which answer is right? The question is malformed. The right question is: what shape does the disagreement have? It is not scatter around a centre; it is two clumps — five people at 2–4 weeks, two at 26–30. That shape says there are two incompatible world-models in the room (most likely: the people who have done this before know about an obstacle the others don't), and every rule that squashes them into one number destroys exactly that information.
The shape of the disagreement is worth far more than its size. Unimodal spread is noise and can safely be averaged away. Bimodal spread means somebody knows something you don't, and averaging it away throws the only warning in the bin.
Write down the aggregation rule before collecting judgments, and write a stopping condition alongside it: if the distribution comes out bimodal, aggregation is forbidden — first go to each clump separately and ask the same question, "what are you assuming that the others might not be?" Stop doing this: seeing the numbers first, then picking the rule that produces the answer you wanted. It is the most common and least self-detectable cheat in the whole aggregation step, because every individual move looks reasonable.
Now the hard part of this section: is there an aggregation rule that preserves independence without destroying disagreement? There is, and no human designed it.
When a honeybee colony swarms, it has to choose a new home. Hundreds of scouts fly out; those that find a candidate site come back and waggle-dance, and the duration of the dance is roughly proportional to how highly that scout rates the site — a better site earns a longer dance, which recruits more scouts to go and look. That step is positive feedback, structurally the same as an ant's pheromone. → Ant colonies and decentralized consensus
But positive feedback alone breaks down: two decent candidate sites each accumulate their own following, neither overwhelms the other, and the swarm stalls halfway. For bees, stalling is death.
Seeley and colleagues reported the missing part in Science in 2012: cross inhibition. A scout dancing for site A delivers a "stop signal" — a head-butt plus a brief vibration — to interrupt a scout dancing for site B, and vice versa. The crucial feature is that the inhibition is across options: you do not reinforce your own side, you suppress the other side's advocates. With that in place, two evenly matched options cannot deadlock, and a tiny advantage gets amplified into a decisive one.
The second part is the quorum: a scout does not count votes globally. It counts how many colleagues are present at the site it is standing on right now. Above a threshold (roughly twenty to thirty bees) it switches to "prepare for takeoff." Seeley and Visscher published a neat confirmation in 2004: artificially delay the formation of the quorum and takeoff is delayed accordingly.
Put those two parts together and what they perform has a name in mathematics: the sequential probability ratio test (SPRT) — the test that reaches a decision on the least evidence for a given error rate. In 2009 Marshall and colleagues placed the bee and ant mechanisms and the evidence-accumulation process in primate brains inside a single decision-theoretic framework and showed both approximate the same optimum.
The point being: a swarm does not vote, does not poll, does not average. It accumulates evidence, inhibits across options, and fires on a threshold. Not one of those three has any counterpart in a standard meeting.
Before designing a group decision process, work out whether you are estimating a quantity or choosing among options. To estimate a quantity, use section 1's machinery: collect independently, take the median or the mean. To choose an option, use the swarm's: make each option's supporters argue at the other option's supporters (across options, not fortifying their own case), and fix an absolute threshold in advance — "when N independent arguments for A have accumulated, A wins" rather than "whoever has more supporters wins." A relative majority can deadlock between two good options; an absolute threshold cannot.
🌀 Literature & the arts · The Oscars changed how they count From the 2009 awards onward, the Academy of Motion Picture Arts and Sciences moved Best Picture from plurality voting back to a preferential ballot: voters rank the nominees, the lowest is repeatedly eliminated and its votes transferred. That rule systematically favours films that most voters quite like and penalises films a small faction adores while the rest recoil — same voters, different rule, different kind of winner. Which yields a method for reading awards: before talking about how "tastes shifted" or "the mood changed" across years, check which counting rule was in force that year. A great deal of what gets read as a shift in collective taste is a shift in the aggregation rule.
By now the three mechanisms — error cancellation, independence, aggregation rule — sound ready to be assembled into a handbook. They aren't. Below are five boundaries, at least two of which are enough to void the whole thing.
One: errors can only cancel if there is no shared bias. The identity in section 1 always holds, but what it computes is squared error relative to the truth. If everyone is wrong in the same direction — same anchor, same dataset, same training — then a large chunk of the first term is common, and the second term cannot catch it, because on that dimension nobody disagrees. In the extreme: a room where everyone believes the same wrong model can have zero diversity, and the crowd error equals that shared error exactly, no matter how many people you add. Crowd wisdom suppresses random error; it does nothing to systematic bias.
Two: when individual accuracy is below chance, more people makes it worse. Condorcet's jury theorem is usually quoted at half length. It says: if each person is independently correct with probability p, then the accuracy of a majority vote rises with group size, converging to 1 — when p > 1/2. The omitted second half: when p < 1/2 it converges to 0. That is, on a class of judgment your team is systematically bad at, scaling up does not dilute the error — it hardens it towards certainty.
🌀 History of medicine · Bloodletting Bloodletting persisted in Western medicine for two millennia with essentially no substantive dissent from inside the profession. Not because physicians were stupid, but because they shared one humoral theory — so on the judgment "does bleeding help?", the profession's p was below 1/2 and its independence was near zero. Condorcet's omitted half then applied in full: more practitioners and stronger consensus meant a more firmly fixed error. Not until 1835, when Pierre Charles Alexandre Louis applied his "numerical method" to the outcomes of pneumonia patients bled and not bled, was the consensus prised open from outside. Which yields a test that still works today: the strength of a professional consensus cannot be used as the strength of the evidence — consensus can come from independent convergence or from a shared prior, and only one question separates them: how independent are these people's information sources?
Three: the identity guarantees less than the marketing implies. "The crowd is never worse than the average member" is an identity, free of charge. "The crowd beats the best member" is not, and often fails. Popular writing routinely runs the two together. Also, the identity holds under squared error — change the loss function (say what you actually care about is whether a red line got crossed, not by how much) and the "subtract the variance" property may no longer be there.
Four: "diversity trumps ability" has drawn a serious technical rebuttal. Hong and Page published a result in PNAS in 2004, popularly summarised as "a randomly chosen diverse team can beat a team of selected high performers." The mathematician Abigail Thompson went through the theorem line by line in the Notices of the AMS in 2014 and argued that, under the conditions where it holds, it is either trivial or cannot support the popular reading. The dispute has never been cleanly resolved. What can safely be borrowed is section 1's identity — it is algebra and cannot be wrong. What cannot be borrowed is the ranking "therefore diversity matters more than ability."
Five: the bee machinery works because interests are perfectly aligned. A colony is one reproductive unit; every scout has the same objective function, which is why "dance longer" can be taken as an honest signal. People are not like that. As soon as participants' payoffs depend on the outcome, any aggregation rule gets played back at you — votes attract strategic voting, ratings attract mutual back-scratching, prediction markets attract hedgers betting against their own beliefs. And this is not an implementation failure: the Gibbard–Satterthwaite theorem proves that with three or more options and unrestricted preference rankings, no voting rule is both non-dictatorial and strategy-proof. "Design a cheat-proof aggregation rule" has no general solution; all you can do is make cheating expensive.
A word on prediction markets. They get treated as the honour student of aggregation rules, and they do have a record: Berg, Nelson and Rietz compared the Iowa Electronic Markets (IEM) against 964 contemporaneous polls across the five US presidential elections from 1988 to 2004, and the market price was closer to the final outcome roughly 74% of the time. Their two advantages are real — they weight by confidence (the surer you are, the more you stake) and they pay for being right (being wrong costs money). But they are just as exposed to boundaries one and five: when every trader reads the same public data, the shared bias survives untouched. And they carry an affliction of their own — the longshot bias, a systematic overpricing of low-probability events, measurable in nearly every real betting market.
Before invoking "there were a lot of us" as a reason, run this checklist; all four must pass. ① Individual accuracy is above chance (p > 1/2). ② Judgments were formed independently (nobody saw anyone else's answer). ③ People are wrong in different ways (no shared dataset, no shared training). ④ The aggregation rule was fixed in advance and matches the problem type (average for quantities, threshold for options). If any one fails, strike "we discussed it and everyone agreed" from the evidence column — the information it carries may be exactly equal to whatever the first speaker said.
Hiring for "culture fit," promoting people who resemble the incumbents, running meetings towards consensus — each is defensible alone, and together they reduce the second term. One possible explanation: the benefit of disagreement lands on the group's accuracy while its cost lands on one particular person's coordination workload. Benefit and cost are booked to different accounts, so whoever pays decides. If that explanation is right, the repair is not to advertise the value of diversity but to merge the two accounts — make whoever answers for accuracy also carry the coordination cost.
Beforehand it is easy (look for channels of mutual observation). Afterwards it is very hard, because both leave nearly the same trace: a lot of people saying the same thing. Three possible handles. Check the timing — a cascade's adoption curve usually has an abnormally steep early jump. Check the variety of reasons — independent converts give wildly different reasons, cascaded ones give near-identical reasons, sometimes in identical words. Check the brittleness — drop in one new piece of evidence and see whether it collapses. The third is the most reliable and can only be used once.
It genuinely cuts off the first round of cascade, but only the first. From the moment discussion opens, everyone can see everyone's answer, and the revision process that follows is cascaded again. Worse, it may manufacture false safety — "we did the independent pre-fill" — leaving later stages more relaxed than they should be. A harder version might make the single disclosure between the independent stage and the aggregation stage irreversible (once written, no edits), at the cost of losing the genuine correction that discussion is supposed to provide. There seems to be a trade-off here that cannot be had both ways, and it is worth working out its exact shape.
Its training corpus comes from hundreds of millions of people, which looks like an enormous aggregation. But by this issue's framework the question is how independent those sources are from one another — and internet text is itself a product of cascades (reposting, cross-citation, shared origins). The sharper layer: once model-generated text flows back into the next round of training data, section 2's machine closes its loop. How the distance between "the majority claim" and "independent evidence" then evolves is measurable and has not yet been measured well.
What a market actually weights is stake size, and stake size = confidence × available capital. For information aggregation, the first factor is what you want and the second is contamination. The standard defence is arbitrage: if the price is off, informed people with money will push it back. But that defence requires "informed" and "wealthy" to be positively correlated, which is exactly the thing needing proof. Play-money markets have performed no worse than real-money ones in some studies; which way that evidence actually falls is worth checking properly on its own.