"Perception is not about seeing truth, it's about having kids" — the theorem is real, but its "Truth" was built to lose

The moment

A beetle is hugging a beer bottle, and the room is laughing.

Hoffman introduces the Australian jewel beetle: dimpled, glossy and brown, the female flightless, the male airborne and "looking, of course, for a hot female." Then he introduces the other species out there in the outback, Homo sapiens — "the male of this species has a massive brain that he uses to hunt for cold beer. (Laughter) And when he finds one, he drains it, and sometimes throws the bottle into the outback." And those bottles are "dimpled, glossy, and just the right shade of brown to tickle the fancy of these beetles." The males swarm them and lose all interest in real females.

Classic case of the male leaving the female for the bottle. (Laughter) (Applause)— Donald Hoffman, TED2015

It is the only point in the talk that earns both a laugh and applause. As the laugh dies down, he continues:

The species almost went extinct. Australia had to change its bottles to save its beetles.— Donald Hoffman, TED2015

Hold on to those two sentences. They are the best piece of theatre in the talk, and they are the first thing section four takes apart — because the original paper contains neither an extinction nor anyone changing a bottle.

The speaker, and that year

Donald D. Hoffman, born 1955 in San Antonio, Texas; BA in quantitative psychology from UCLA in 1978; PhD in computational psychology from MIT in 1983. His two advisors were David Marr and Whitman Richards.

That detail is worth a pause. Marr's Vision (1982) is the book that turned vision-as-reconstruction — recovering the three-dimensional structure of the world from a two-dimensional array of light intensities — into the working language of an entire discipline; every computational-vision course still starts there, and Hoffman's own papers still cite Marr 1982 dutifully. What his student went on to do was stand up and say that the reconstruction reconstructs nothing — that the "external world" we recover is an interface, not a world.

He has taught in UC Irvine's Department of Cognitive Sciences since 1983 (now professor emeritus), wrote Observer Mechanics (1989) and Visual Intelligence, and in 2019 published the trade book The Case Against Reality.

When he walked onto the TED2015 stage in March 2015 he was standing in a very particular spot. The master statement of the view, written with Manish Singh and Chetan Prakash, was still in press; "The Interface Theory of Perception" appeared that December in Psychonomic Bulletin & Review volume 22, pages 1480–1506 — and not on its own, but as open peer commentary: nine commentaries from perception scientists and philosophers, plus twenty-five pages of the authors' point-by-point replies. In other words, he delivered the conclusion to several million people nine months before the biggest academic firefight of his career began. In the hall sat the TED audience; the man who came up to take questions was TED's own Chris Anderson. The talk now has 5.62 million plays on TED's site.

Glossary

The talk, in full

He opens with an admission: he is fascinated by "the greatest unsolved mystery in science," and it is "personal." What is the relationship between your brain and your conscious experiences — the taste of chocolate, the feeling of velvet? He quotes Thomas Huxley in 1868: that a state of consciousness should come about "as the result of irritating nervous tissue" is "just as unaccountable as the appearance of the genie when Aladdin rubbed his lamp." Huxley knew brain activity and conscious experience were correlated; he did not know why. A century and a half on, we know an enormous amount about brain activity and the relation is still a mystery.

Why so little progress? Some experts say we simply lack the concepts and the intelligence: "We don't expect monkeys to solve problems in quantum mechanics, and as it happens, we can't expect our species to solve this problem either." He disagrees, and he is more optimistic: we have simply made a false assumption. Fix it and the problem might give. That is the frame for the next twenty minutes — name the assumption, show it is false, say how to fix it.

His entry point is almost domestic. I open my eyes and have an experience I describe as a red tomato a meter away; so I come to believe that in reality there is a red tomato a meter away. I close my eyes and the experience becomes a grey field — but is the tomato still there? "I think so, but could I be wrong?"

His reason for asking is history. We have misread our perceptions before: we thought the Earth was flat because it looks that way, and that it was the unmoving centre of the universe, again because it looks that way. Wrong both times. Then Galileo took the thought one step further:

I think that tastes, odors, colors, and so on reside in consciousness. Hence if the living creature were removed, all these qualities would be annihilated.— Galileo, as quoted by Hoffman on stage, TED2015

Then comes the hardest-working stretch of the talk. Neuroscientists tell us about a third of the brain's cortex is engaged in vision; simply open your eyes and look around the room and billions of neurons and trillions of synapses are engaged. That is a little surprising, because insofar as we think about vision at all we think of a camera. Part of vision really is camera-like: a lens focuses an image onto 130 million photoreceptors, so the eye is a "130-megapixel camera." But a camera does not need billions of neurons. What are those neurons doing?

Creating, in real time, every shape, object, colour and motion you see. "It feels like we're just taking a snapshot of this room the way it is, but in fact, we're constructing everything that we see" — and not the whole world at once, just what is needed in the moment. He shows two demonstrations: red discs with bits cut out of them that, rotated slightly, make a three-dimensional cube pop out of the screen — the screen is flat, so the cube "must be your construction" — and glowing blue bars with sharp edges sweeping across a field of dots, where "in fact, no dots move" and all he changes frame to frame is the dots' colour between blue and black.

No perception scientist will quarrel with any of that. The disagreement begins with the next sentence: the textbooks say we do not merely construct, we reconstruct — the "red tomato" experience "is actually an accurate reconstruction of the properties of a real red tomato that would exist even if I weren't looking." On what grounds? Evolution. Ancestors who saw more accurately had a competitive advantage and were likelier to pass on their genes; we are their offspring. He quotes the standard textbook line:

"Evolutionarily speaking, vision is useful precisely because it is so accurate."— standard textbook line, as quoted by Hoffman on stage, TED2015

Then he starts dismantling that step, first with the beetle on the bottle (see above). The males had found females successfully for thousands, perhaps millions of years: "It looked like they saw reality as it is, but apparently not. Evolution had given them a hack. A female is anything dimpled, glossy and brown, the bigger the better. (Laughter) Even when crawling all over the bottle, the male couldn't discover his mistake." And if you object that beetles are simple creatures but surely not mammals — "Mammals don't rely on tricks. Well, I won't dwell on this, but you get the idea. (Laughter)"

With the room warmed up, he turns it into a technical question: does natural selection really favour seeing reality as it is? "Fortunately, we don't have to wave our hands and guess; evolution is a mathematically precise theory." Hence the steak. For a hungry lion looking to eat, it enhances fitness; for a well-fed lion looking to mate, it doesn't; for a rabbit in any state, it doesn't. So fitness depends on reality, yes, but also on "the organism, its state and its action." Fitness is not the same thing as reality, and it is fitness that figures centrally in the equations of evolution.

So he reports the result: hundreds of thousands of evolutionary game simulations, with many different randomly chosen worlds and organisms competing for resources — some seeing all of reality, some part of it, some none of it and tuned only to fitness. Who wins?

Well, I hate to break it to you, but perception of reality goes extinct. In almost every simulation, organisms that see none of reality but are just tuned to fitness drive to extinction all the organisms that perceive reality as it is.— Donald Hoffman, TED2015

The rest of the talk makes that counterintuitive result liveable. The metaphor is the desktop. The icon for the TED talk you're writing is blue, rectangular, lower right — does that mean the text file inside the computer is blue, rectangular and in the lower right-hand corner of the computer? Of course not; anyone who thought so "misinterprets the purpose of the interface. It's not there to show you the reality of the computer. In fact, it's there to hide that reality." And so: "Space and time, as you perceive them right now, are your desktop. Physical objects are simply icons in that desktop."

He raises three objections against himself. The sharpest comes first: Hoffman, if that train coming down the track at 200 MPH is just an icon, why don't you step in front of it?

I wouldn't step in front of that train for the same reason that I wouldn't carelessly drag that icon to the trash can: not because I take the icon literally -- the file is not literally blue or rectangular -- but I do take it seriously. I could lose weeks of work.— Donald Hoffman, TED2015

If you see a snake, don't pick it up; if you see a cliff, don't jump off. "That does not mean that we should take them literally. That's a logical error." Second objection: physicists have long said the train's metal only looks solid and is mostly empty space, so there's nothing new here. His answer: that is like pulling out a magnifying glass, seeing pixels in the blue icon and concluding that those are the reality of the computer — no, you are still on the desktop, and "those microscopic particles are still in space and time: they're still in the user interface." Third: we all see the train, so none of us can be constructing it. He goes back to the cube: "We all see a cube because we all, each one of us, constructs the cube that we see."

And then he follows the inference all the way down, in what is the least-quoted and most quotable passage in the talk:

And here's the kicker: When I have a perceptual experience that I describe as a brain, or neurons, I am interacting with reality, but that reality is not a brain or neurons and is nothing like a brain or neurons. [...] Brains and neurons have no causal powers. They cause none of our perceptual experiences, and none of our behavior.— Donald Hoffman, TED2015

So what is reality? He offers two options: perhaps it is "some vast machine that causes our conscious experiences" — "I doubt this, but it's worth exploring" — or perhaps "some vast, interacting network of conscious agents, simple and complex, that cause each other's conscious experiences," which "isn't as crazy an idea as it seems, and I'm currently exploring it." Then the dare:

Dare to recognize that perception is not about seeing truth, it's about having kids. And by the way, even this TED is just in your head.— Donald Hoffman, TED2015

Chris Anderson walks on and opens with "If that's really you there, thank you." Then he asks the question everyone should ask: if evolution does not favour reality, doesn't that to some extent undermine all our ability to think we can think the truth — possibly even including your own theory? Hoffman's answer has two layers. First, science carries on: what turned out false is one theory, that perception is like reality and reality is like our perceptions; throw it away and go on postulating other theories — "it's actually progress." Second, a concession almost nobody noticed:

The evolutionary game simulations that I showed were specifically about perception, and they do show that our perceptions have been shaped not to show us reality as it is, but that does not mean the same thing about our logic or mathematics. We haven't done these simulations, but my bet is that we'll find that there are some selection pressures for our logic and our mathematics to be at least in the direction of truth.— Donald Hoffman, answering Chris Anderson, TED2015

That is: by his own account the same evolutionary argument has never been run on logic or mathematics, and his bet is that the answer there comes out the other way. The concession matters later.

Under the iceberg

"Hundreds of thousands of simulations" and "almost every simulation" become, on paper, a theorem with a name, an inequality, and a good many premises.

Here is the theorem. Prakash, Stephens, Hoffman, Singh and Fields, "Fitness Beats Truth in the Evolution of Perception" (Acta Biotheoretica 69: 319–341, September 2021):

Over all possible fitness functions and a priori measures, the probability that the Fitness-only perceptual strategy strictly dominates the Truth strategy is at least (|X| − 3)/(|X| − 1), where |X| is the size of the perceptual space.— Prakash, Stephens, Hoffman, Singh & Fields, "Fitness Beats Truth," Theorem 4 (the original uses a math-italic X)

Put numbers in and you can see what the formula says. With a perceptual space of 100 states the lower bound is about 98%; with 10 states, 78%; with 4 states, 33%; with 3 states, zero. The stage version's "almost every simulation" is an asymptotic result that holds once the perceptual space is large — which is not itself a flaw, since the human perceptual space is obviously enormous, but it is not the same thing as the unconditional "evolution does not favour veridical perception."

What matters more is what "Truth" is in the proof. Both strategies share one sensory map p from world states w to sensory states x, and every x corresponds to a whole bundle of world states that could have produced it (the paper calls it a fibre). The Fitness-only strategy picks the x with the greatest expected fitness over the whole fibre. The Truth strategy instead computes, for each x, the maximum-a-posteriori world point w*, then compares the fitness f(w*) of those single points and takes the best. Which is why the paper's own proof contains this line:

On the other hand, the fitness payoff to the Fitness-only strategy is, by definition the maximum expected fitness F(x_F) over all fibers, so clearly, F(x_T) ≤ F(x_F).— ibid., proof of Theorem 4 (subscripts transliterated from the PDF's math symbols: F(x_F) is the Fitness-only payoff, F(x_T) the Truth payoff)

"Clearly" because it is definitional. The "Truth" in this theorem is not a strategy that sees the world accurately and then acts well; it is a strategy forbidden from averaging fitness the way its opponent does — it may only stare at one point. That an opponent who understands the world but cannot compute an expected payoff loses to an opponent who computes nothing but expected payoff is true; it is also some distance from "seeing reality confers no evolutionary advantage."

Hoffman's own wording on paper is much narrower than on stage. In his review article in Current Directions in Psychological Science, the same conclusion reads:

The problem is not that veridical perceptions are necessarily counter-adaptive, but rather that veridicality is irrelevant to adaptation, meaning that veridicality per se contributes nothing when reward value is varied orthogonally to it. One can, of course, construct payoff matrices in which reward and veridicality are correlated. But such correlations are not generic, in the mathematical sense that the unbiased probability of their occurrence is near zero.— Hoffman, "The Interface Theory of Perception"

On stage: veridical perception goes extinct. On paper: veridicality is irrelevant, and that is said under the condition that reward value varies orthogonally to it. These are not two phrasings of one sentence.

He also states a hard limit of his own — one that blocks his positive programme as well. In the same review, answering the objection that the lawful way our percepts change as we move must reveal something about the structure of the world, he invokes an "Invention-of-Symmetry Theorem" proved by Chetan Prakash and writes: "The theorem only permits one to infer a lower bound on the cardinality of states of the objective world, but not to infer anything at all about its structure." If the mathematics licenses only a lower bound on a cardinality, then it equally fails to license "reality is an interacting network of conscious agents." The hammer that knocks the house down cannot build one. A related note: that review's funding statement is explicit — the Federico and Elvia Faggin Foundation, which funds consciousness research and leans toward consciousness being fundamental.

Now back to the beetle. The primary source is D. T. Gwynne and D. C. F. Rentz, "Beetles on the bottle: male buprestids mistake stubbies for females (Coleoptera)," Journal of the Australian Entomological Society volume 22, pages 79–80 — two pages. The observation was made in September 1981 near Dongara, Western Australia: males flying low, settling on discarded 375 ml beer bottles (stubbies). They ran controls, laying out bottles of various colours, sizes and shapes; only the stubbies drew beetles, and wine bottles of a slightly different brown drew none. Gwynne and Rentz were awarded the 2011 Ig Nobel Prize in biology for it. Rentz's account adds a detail the stage version leaves out: ants "were actively pulling the love-starved beetles to the ground and dismembering them" — what actually kills them is not the mistake itself but staying put after making it. The species is now called J. saundersii.

Rentz, the co-author, wrote the sequel on his own blog. On the bottles:

By the way, we contacted the Swan Brewing Company suggesting they modify their bottles so they are not attractive to the beetles but never received an answer.— David Rentz, "Beetles on the Bottle," BunyipCo

He goes on: the "tubercles" did eventually disappear from beer bottles, and that, together with the deposit Australia places on bottles, "may have helped conserve the dwindling Western Australian Jewel Beetle fauna" — note the "may have," and note that it is incidental, not Australia changing its bottles to save a beetle. As for extinction:

By far their worst enemy is not from mistaken sexual experiences but from development-especially the destruction of their habitat for agriculture and/or mining.— David Rentz, ibid.

So both halves of the factual claim behind the laugh line are wrong: no near-extinction, and no country changing its bottles. In fairness, Hoffman is far more restrained about the beetle in print — Martínez quotes his 2015 paper at page 1481, where the claim is only that the beetles' perceptions "relied not on veridical information but rather on heuristics that worked in the niche where they evolved." What got amplified is the stage version, not the paper version.

What became of it

December 2015: the firefight arrived on schedule. Psychonomic Bulletin & Review volume 22 published the main paper (pages 1480–1506) alongside nine commentaries, by Barton Anderson, Jonathan Cohen, Shimon Edelman, Jacob Feldman, Chris Fields, Jan Koenderink, Rainer Mausfeld, Brian McLaughlin and E. J. Green, Zygmunt Pizlo, and Matthew Schlesinger. The titles alone carry the tone: Pizlo's is called "Philosophizing cannot substitute for experimentation." Hoffman and colleagues replied across pages 1551–1576, opening by marking their opponent precisely: "Current textbooks assert, to the contrary, that perception is useful because, in the normal case, it is veridical. Intuition, both lay and expert, clearly sides with the textbooks."

The sharpest cut was made with his own lion. Barton Anderson's "Where does fitness fit in theories of perception?" (same volume, 1507–1511) seizes Hoffman's key premise — that fitness functions are generally non-monotonic — and draws the opposite conclusion from it. His abstract: "I argue that the ubiquity of nonmonotonic payoff functions requires that (1) perception tracks 'truth' for species that adapt on ontogenetic time scales and (2) that perception should be distinct from utility. These conditions are required to pursue an adaptive strategy to mitigate homeostatic imbalances." In plain terms: precisely because the same steak is worth something to a hungry lion and nothing to a sated one, an animal that adjusts its behaviour within its own lifetime cannot encode "steak" directly as "points." It has to represent "there is meat over there" separately from "do I want it now," or it could never change its mind after dinner. The example Hoffman used on stage to show that fitness ≠ reality is the same example that shows perception has to be kept distinct from utility.

Rainer Mausfeld's cut goes both ways. His title is his position: "Notions such as 'truth' or 'correspondence to the objective world' play no role in explanatory accounts of perception" (same volume, 1535–1540). Half of his abstract agrees: "I argue, partly in accordance with their objective, that issues of 'truth' or 'veridicality' have no place in explanatory accounts of perception theory, and rather belong to either ordinary discourse or to philosophy." That is to say, the target Hoffman attacks with full force is largely a sentence in the textbooks rather than the working science. The other half he refuses: "I regard, however, their general presumption that the evolutionary development of core achievements of the human perceptual system would be primarily determined by aspects of fitness and adaption as unwarranted in light of the evidence available."

In 2019 someone built a counterexample out of a different beetle. The philosopher Manolo Martínez published "Usefulness Drives Representations to Truth — A Family of Counterexamples to Hoffman's Interface Theory of Perception" in Grazer Philosophische Studien 96: 319–341. He rebuilds Hoffman's model with one change: the organism decides using two complementary cues. Bark beetles choosing a host tree need both the right semiochemical and the right bark colour; neither cue alone settles whether the tree will do, and together they do. In that model the sender strategy with the highest expected payoff becomes the one that reports the cue faithfully. His conclusion: "Hoffman's skeptical argument to the effect that usefulness trumps truth only works if the signals produced by the sender are the sole available cue in the maximization of usefulness." He also notes in passing that the evolutionary-game dressing is doing no work — because in these models dominant strategies always go to fixation, "the evolutionary game-theoretic spin is, as far as I can see, theoretically idle." And he relays an observation from Jonathan Cohen's 2015 commentary: Hoffman's description of the beetle case is tendentious, since a correct theory of content might well conclude that the beetle is correctly representing the presence of a shiny brown surface, rather than incorrectly representing a female.

His own line has kept moving too. In 2020 he and Prakash, Fields, Prentner and Singh published "Fact, Fiction, and Fitness" in Entropy 22(5): 514, proving four more theorems in the same direction: fitness payoff functions are generically not homomorphisms of key perceptual structures. In 2021 the FBT theorem landed formally in Acta Biotheoretica. The argument also moved into philosophy journals: in 2023 Jeffrey N. Bagwell published "Debunking interface theory: why Hoffman's skepticism (really) is self-defeating" in Synthese volume 201 — a paper this run could not read in full, so it is reported here only as a bibliographic fact.

And his positive claim is still where it was. "A network of conscious agents" was, in 2015, the thing he was "currently exploring." Eleven years on it remains a mathematical construction that has yielded no testable prediction in perception science — a very different position from the theorem half. More subtly, the concession from the stage still hangs there: he said himself that the simulations were run on perception, never on logic or mathematics, and that he would bet logic and mathematics are pushed "at least in the direction of truth." Yet his entire argument is driven by mathematics. If evolution biases us toward truth in mathematics, why would it fail so completely to do so in perception? That asymmetry needs a reason, and so far the reason is a bet.

Finally, the dissemination. 5.62 million plays have made this talk the most frequently cited source for claims of the form "science has proved reality is an illusion." What got remembered is the beetle and the desktop — not (|X| − 3)/(|X| − 1), not the fact that "Truth" is not allowed to take an expectation, and certainly not Hoffman's own written sentence that veridicality is irrelevant to adaptation. He said on stage that we are blind to our own blindnesses; the neatest footnote to the talk is that the version of it that travelled proves the point.

Sources

• Official TED transcript: Do we see reality as it is?The sole source for every talk quotation on this page (including the Chris Anderson Q&A) and every number spoken on stage; pulled with tools/ted_transcript.py and read end to end, ~3,190 words.
• Hoffman, "The Interface Theory of Perception" (Current Directions in Psychological Science version, author's copy)His own written statement of the view. The "veridicality is irrelevant to adaptation" qualification, the Invention-of-Symmetry limit to a cardinality lower bound, and the Faggin Foundation funding statement all come from here.
• Prakash, Stephens, Hoffman, Singh & Fields, "Fitness Beats Truth in the Evolution of Perception" (author's copy; published in Acta Biotheoretica 69: 319–341, 2021)The theorem itself. The (|X| − 3)/(|X| − 1) bound, the definitions of the two strategies, and the proof's line "so clearly, F(x_T) ≤ F(x_F)" support the whole iceberg section and the diagram.
• Barton L. Anderson, "Where does fitness fit in theories of perception?" (Psychonomic Bulletin & Review 22: 1507–1511, 2015)The sharpest of the nine peer commentaries, turning non-monotonic payoff functions into an argument that perception must track truth and be distinct from utility. Only its abstract is quoted here; the full commentary was not read.
• Manolo Martínez, "Usefulness Drives Representations to Truth" (Grazer Philosophische Studien 96: 319–341, 2019)A formal counterexample independent of Hoffman, read in full. The two-cue bark-beetle model, the "only for cue-driven agents" conclusion, the footnote calling the game-theoretic spin "theoretically idle," and Cohen's 2015 objection about the beetle's content all come from here.
• David Rentz, "Beetles on the Bottle" (BunyipCo)The 1983 paper's co-author in his own words, used to check the beetle on stage: the bottle controls, the letter to the brewery that "never received an answer," the ants, and habitat destruction as the real threat.