TOPIC 1 · PHASE A COGNITION

Perception as Inference

You're not "seeing" — you're guessing

2026-07-10 · BigCat

You think you're seeing the world — really you're guessing, and the world just corrects you.

Right now you're staring at a screen. Feels pretty direct, right — light hits your eyes and the brain "sees" it. But decades of neuroscience have flipped that intuition inside out: your brain isn't a camera that faithfully snapshots the world; it's a machine that's constantly guessing what the world looks like, then using the signal your eyes send back to correct itself. Every frame you see is one of the brain's "best guesses." This issue we chew on just that one thing — but once you believe it, everything after it (attention, memory, even consciousness) starts to loosen.

// 01

The brain is a "guessing machine," not a camera

Start with the old intuition that got overturned: photons hit the eye → the signal travels up layer by layer → the brain finally "sees." Sounds obvious. But anatomy debunked it long ago: along the brain's visual pathway, the connections running backward, from "high" levels down to "low" ones, vastly outnumber the ones running up from the eyes. In other words, most of the brain's effort isn't spent receiving data — it's spent sending guesses downward.

Higher brain · forms a "guess" first Eyes · send back only "misses" Prediction · bulk Error · trickle
The main direction of perception is "brain → eyes" prediction, not "eyes → brain" data

Don't buy it? Look at these two gray squares below. The left one sits on a light background, the right one on a dark one — and you're almost certainly sure the right one is brighter. But the two are exactly the same color:

Light background Dark background Both grays have identical fill · #8a8a8a
The lightness you "see" is guessed by the brain from context, not measured by the eye

This isn't your eyes malfunctioning — it's the brain actively "filling in" for you, using its experience that "background affects objects." So what you see is never raw pixels; it's the result after being shaped by the brain's expectations. (For how the path from eye to brain runs, station by station: Visual pathway)

AI cross-read

AI has a whole family of things called generative models (image-makers like Stable Diffusion, the AI face-swap tools, all belong here), and the idea is exactly the brain's: rather than hard-storing a pile of pixels, learn an internal model that can "generate" the world, then use it to explain what's in front of you — propose a hypothesis, generate a version, compare it to reality, correct by the error. The brain's "guess first → send down → correct with the senses" runs on the very same skeleton.

// 02

Only the "misses" are worth reporting

How does the brain guess efficiently? Through a very frugal trick: each layer passes only the "part it got wrong" upward. Picture the brain as a stack of floors, each one predicting what will happen on the floor below and sending that prediction down; the lower floor compares the prediction against what actually happened, and only the mismatches (the errors) travel back up. The parts it guessed right never go up at all — because they carry no news.

Higher layer Middle layer Eyes / lower layer prediction↓ error↑ hit = silence · miss = report
Hierarchical prediction: predictions go down, only "surprises" go up

This one move explains a lot of everyday little things. You can't hear the hum of your own fridge — because it's too predictable, its error is zero, so it simply isn't reported; yet the moment it stops, you snap to attention, because "the prediction failed." The more familiar something is, the more invisible it becomes — not because your ears got duller, but because the brain already guessed it and subtracted it in advance. At bottom, the brain's attention is reserved for the unexpected.

AI cross-read

AI keeps reaching for that same "only record the misses" thrift — don't repeat the known, store only the increment. The tightest example is training a large language model (the ChatGPT kind): at every step the model guesses "what's the next word," and how badly it's off (that number is called the loss) is its "prediction error," after which it nudges itself only in "the direction it got wrong." The brain uses error to correct its judgment in real time; the model uses error to correct itself slowly — both are, at heart, "being pushed to learn by surprise."

// 03

The dimmer the light, the more you "see what you expect"

Put the first two points together and you get a wonderfully practical rule: what you finally see = the brain's expectation + the evidence in front of you, weighted by whichever is more trustworthy.

The crux is "which is more trustworthy." In dim light, the evidence your eyes send back is very unreliable, so the brain leans harder on its own expectation — and that's how you mistake a coat hanger for a crouching figure, or the wind for someone calling your name. Flip it around: in broad daylight, with plenty of evidence, expectation has to yield and go with what's actually seen. This also hands us a precise definition of "attention": attention is the brain temporarily raising the trustworthiness of the evidence in some spot, giving it more standing to rewrite the world in your eyes (attention itself is Topic 2).

AI cross-read

In AI this is called weighting by reliability: when evidence is thin, trust your experience (the prior) more. AI models have their own knob for "how much to trust the prior" (guidance in image generation, temperature in chat models are of this kind), except the brain turns that knob with a quantity called precision, per sense, in real time, automatically. Even better — an AI "hallucination" and a human illusion are the same story: the prior is over-trusted, the evidence gets overpowered, and the brain or model "sees" something that isn't there at all.

// 04

One more move: don't change the guess, change the world

When prediction and reality don't match, what do you do? Everything above was about changing the brain — updating the guess to fit the world. But there's a symmetric route: changing the world to fit the guess. This is active inference, and it's the most beautiful part.

Take the simplest case: you predict "my eyes should be pointed at the center of the desk," but right now they aren't — so you move your eyes to actually aim at the center. The error is erased, not by revising a belief but by an action. Push this all the way and you get a startling restatement: every action of yours is really a "prediction the body makes come true." When you reach for a cup, the brain first predicts "the hand is already gripping the cup," and the error from that not-yet-realized prediction drives the muscles to make it real.

So perception and action are really two outlets of the same machine: perception asks "what is the world like," action asks "how do I make the world become what I guessed." That's why later issues on movement, on decision-making, even on meditation (actively tuning your own predictions) all hook back onto this same main line.

AI cross-read

This resembles reinforcement learning, the mainstream way AI trains agents, but with an intriguing difference: reinforcement learning has the agent "maximize reward," while active inference has it "minimize prediction error" — often the two are equivalent, but the latter folds "exploring the world" and "achieving a goal" into one and the same thing. It's inherently the sort of agent that "builds a world model in its head first, imagines with it, then acts to make the imagination come true" — an approach the AI field is now rediscovering, and one the brain has used for hundreds of millions of years.

🌀 Crossing over · interdisciplinary echoes

"You have never touched the world directly — only the model the brain builds" — several ancient traditions arrived at this conclusion thousands of years ago; they just didn't have fMRI:

// Going deeper

If what I "see" is forever the brain's guess, what gives me the right to trust that the world I see is real?
Strictly, you've never "directly" touched the world — what you touch is always the brain's model. But this doesn't collapse into doubting everything: that model is tethered to reality by continuous error, and guessing wrong carries a cost (you walk into a wall, you grasp at nothing). So perception isn't a hallucination — it's a "controlled hallucination" constantly recalibrated by reality — real enough to keep you alive, but never pixel-level truth. This also explains why two people can "see" different things: different expectations, different results.
If "familiar = zero error = no sensation," then why do we still love repetition — replaying a song, rewatching a film?
Because "error" isn't a single layer. The low layer (the notes, the frames) has been guessed, but the high layer (the arc of the emotion, the meaning, the resonance with memory) is still throwing off fresh error; and "perfect predictability" can itself become a kind of safety (rituals, white noise, ASMR run on this). The sweet spot of aesthetics tends to fall between "a skeleton you can predict + just the right amount of surprise" — too surprising is noise, too predictable is boredom. This thread comes back in Topic 19 on awe, and in music cognition.
If the brain and generative models share one skeleton, then are an LLM's "hallucinations" and a human's "illusions/delusions" the same thing?
Mechanistically they're deeply homologous: the prior is over-trusted, the evidence gets overpowered, and both sides "generate" details that aren't there. But there's a key difference — a healthy brain has an extra leg: active inference. It can move, sample new evidence to test its guess (take another look, reach out and touch). Most of today's LLMs lack that "go and check" loop and can only spin further within existing priors — which also points to how to mitigate hallucination: give the model the ability to fetch evidence, and let that evidence outweigh the prior. Tool calls and retrieval are, in essence, giving the model "senses."

// Further reading