中文 →
Blog Deep-Read · 03 · astralcodexten.com

Scott Alexander · Astral Codex Ten
Why evidence stops working: a psychiatrist's craft of doubt

2013–2026 · predecessor Slate Star Codex, 1,557 posts (he deleted them all once, in 2020) · current site Astral Codex Ten, 1,400+ posts · this page scraped and parsed 870 of them, ~3.4M words · still active: latest post 1 September 2026

There is a security guard who works in a very boring building. It basically never gets robbed. He sits in his booth doing the crossword. Every so often there is a noise, and he checks whether it is robbers or just the wind.

It is the wind. It is always the wind. It is never robbers. Nobody wants to rob the Pillow Mart in Topeka. If a building gets robbed once every decade or two, he might go an entire career without meeting a real one.

So he develops an extremely useful heuristic: when you hear a noise, ignore it and keep crossing words. It's just the wind, bro.

The heuristic is right 99.9% of the time. As heuristics go, that is a very good score. The only problem is:

He could be losslessly replaced by a rock with the words "THERE ARE NO ROBBERS" on it. Heuristics That Almost Always Work, 2022

The man who wrote that is Scott Alexander — real name Scott Siskind, a practising psychiatrist in the Bay Area. He started blogging in 2013. In 2020, fearing that the New York Times was about to print his surname, he deleted all 1,557 of his posts. In 2021 he restarted on Substack, and then, in the last line of the very first post of the new blog, printed his own real name himself.

He writes about medicine, statistics, politics and AI at a volume no reader can keep up with. But underneath those 1,400-odd posts there is only ever one question:

What makes you think you know anything?

He has an occupational reason to ask. A psychiatrist spends his days sitting across from someone whose reasoning machinery is demonstrably broken. The phobic patient knows perfectly well that this dog will not bite him. Lock him in a room with a gentle Rottweiler and he comes out more afraid of dogs than when he went in.

The post Scott links back to more than any other — I counted the internal links across all 870 posts I scraped, and it is cited by 16 later posts, nearly double the runner-up — is about exactly how that machine jams. He calls it a trapped prior.

The brain does not perceive first and judge second. It blends raw experience with context and hands you a finished product that feels exactly like raw experience. Dye a white wine red and many people taste red wine. A phobia is that mechanism at its limit: the prior is strong enough to colour the experience itself, so every good encounter with a dog gets processed as a frightening one, which then feeds the prior and makes it stronger.

Then he pushes it into politics, and gets a genuinely alarming sentence:

they start out wrong, and each new fact they learn, each unit of effort they put into becoming more scientifically educated, just makes them wronger. Trapped Priors As A Basic Problem Of Rationality, 2021

This is not a figure of speech. In a 1979 experiment, partisans on both sides of capital punishment read two studies, one pro and one con, deliberately designed to be equally well conducted. Both groups came away more certain they had been right all along. More evidence, wider disagreement — which he treats as the diagnostic sign of a prior that has become trapped.

If that were all, Scott would just be a gloomy populariser of cognitive bias. What is actually original is the other half: you can still know things — you just have to know how to listen.

He makes the point with an example designed to annoy his own side. Say you are a liberal who does not trust FOX News, and you are at the airport when FOX reports a mass shooting. Do you believe it? He does. Because:

The level on which FOX News is bad isn't the level where they invent mass shootings that never happened. Bounded Distrust, 2022

He calls this bounded distrust. A biased source is not random noise; it is playing a game with rules. There are lines it will cross and lines it will not, and everyone more or less knows where they are. Learn the rules and you can reliably extract true information from a source that really is trying to mislead you.

The skill has a cruel corollary. Savvy people almost never experience the government lying to them. The Soviet government announces a good harvest; the savvy notice it said "good" rather than "glorious" and correctly conclude that everyone is about to starve. To the savvy, the people who believed "good harvest" and then went hungry — and became conspiracy theorists afterwards — look paranoid. To those people, the savvy look credulous.

Scott's reconciliation is the best thing in the essay:

The savvy people need to realize that the clueless people aren't always paranoid, just less experienced than they are at dealing with a hostile environment that lies to them all the time. Bounded Distrust, 2022

So the core move of this mind is: distrust is not a switch, it is a craft. It can be done well or badly, and both ways of doing it badly — believing everything, believing nothing — get you replaced by that rock.

His own most honest line is buried in the last paragraph of the trapped priors essay. Ironically, he says, his prior on the theory of trapped priors is itself trapped — everything he reads makes him more convinced it is true. He leaves that sitting at the end of the piece, and adds: I look forward to getting outside perspectives.

For why he calculated that the screening scan for donating a kidney was fifteen times deadlier than the surgery, how the Cult Of The Rock replaced every expert on a volcanic island, how 29 ivermectin studies came down to worms, and which points he conceded after a man named Alexandros Marinos wrote 21 posts proving him wrong — switch to "Full" above.

Updates