Day 27 · 2026.07.19

Probability Paradoxes

Four scenes where intuition crashes hardest against probability
"Probability theory is nothing but common sense reduced to calculation." — Laplace; these paradoxes are exactly where common sense and calculation quarrel.

The Monty Hall Problem

Information itself is probability
Conditional Probability
Intuition

Three doors: one hides a car, two hide goats. You pick door 1. The host—who knows where the car is—opens a goat door, say door 3, then asks: switch to door 2, or stick with door 1?

Nearly everyone blurts out "two doors left, 50/50, it makes no difference." Wrong. Switching wins with probability $2/3$. One sentence cracks it open: your first pick lands on the car only $1/3$ of the time and misses it $2/3$ of the time. And whenever you missed (the $2/3$ case), the host is forced to dodge the car and open the one remaining goat door—so the door left standing must be the car. "Switch" therefore bets on "I picked wrong to begin with," an event whose probability is exactly $2/3$. The host is not opening a door at random; his move leaks information.

You pick door ① · host opens a goat door · switching wins 2/3 car at ① (1/3) car goat goat switch→lose car at ② (1/3) goat car open switch→win car at ③ (1/3) goat open car switch→win two of three equally likely cases reward switching → P(switch wins)=2/3
Formal definition

Let the car sit behind door $i$, you pick door 1, the host opens door 3. By Bayes: $P(\text{car at }2\mid\text{opens }3)=\dfrac{P(\text{opens }3\mid\text{car at }2)P(\text{car at }2)}{P(\text{opens }3)}=\dfrac{1\cdot\frac13}{\frac12}=\dfrac23$. In the numerator $P(\text{opens }3\mid\text{car at }2)=1$—if the car is at 2 he can only open 3; whereas if the car is at 1 he opens 2 or 3 with equal chance. That asymmetry is what manufactures the $2/3$.

Why it's beautiful

Scale it to 100 doors and it becomes obvious: you pick door 1 (a $1/100$ shot), the host flings open 98 goat doors and leaves only yours and one other. The door he deliberately avoided concentrates the entire probability of the other 99. Intuition fails not at the arithmetic but at ignoring the condition "the host knows"—information changes probability. This is the seed of the Bayesian worldview: probability is not a property of objects but a measure of how much you know.

Applications

This is Bayesian updating in miniature. In medical diagnosis, how one test result reshuffles disease probabilities is the very same computation; so are spam filtering, the posterior of an A/B test, and a robot's SLAM tightening its position distribution as sensor readings arrive. It also works as a diagnostic mirror: many "statistical intuition failures" stem from treating an informative observation as a random one.

Question to ponder
If the host does not know where the car is, opens a door at random, and it happens to be a goat—does switching still help? The answer resets your sense of where information comes from: now switching and staying are each $1/2$. The same "a goat door got opened" is worth a world of difference depending on whether it was intentional or accidental.

The Birthday Problem

Collisions that grow quadratically
Combinatorics
Intuition

Put just 23 people in a room and it is already more likely than not that two share a birthday. Twenty-three feels absurdly few—there are 365 days in a year.

What fools you is the wording. You instinctively think "someone shares my birthday," which is indeed hard. But the question asks about "any two people." Twenty-three people form $\binom{23}{2}=253$ pairs, and each pair is one chance for a match. With 253 chances thrown at 365 slots, a hit is no surprise. The point: the quantity to compare is not the number of people but the number of pairs—and that grows with the square of the people.

Formal definition

It is easier to count "all distinct" and subtract: $P(\text{no repeat})=\dfrac{365}{365}\cdot\dfrac{364}{365}\cdots\dfrac{365-n+1}{365}$, so $P(\text{at least one pair})=1-\text{that}$. For modest $n$ there is a clean approximation $P\approx 1-e^{-n(n-1)/730}$—the $n(n-1)/2$ in the exponent is precisely the number of pairs, and $730=2\times365$. Plugging in $n=23$ gives $\approx 0.507$.

Why it's beautiful

It puts "combinatorial explosion" on full display: collision chances scale as $\sim n^2/2$, not $n$. So to make some coincidence more likely than not, the sample size you need is about the square root of the number of slots—$\sqrt{365}\approx 19$, the same order as 23. This "$\sqrt{N}$ threshold" is not special to birthdays; it is the universal rhythm of every "throw balls into $N$ buckets at random" problem. It feels wrong because the human mind extrapolates linearly and is unprepared for quadratic growth.

Applications

It is the heart of the cryptographic birthday attack: to find a hash collision you need not try all $2^{n}$ inputs, only about $2^{n/2}$—which is why a 128-bit hash offers only 64 bits of collision resistance, and the mathematical root of MD5's downfall. The same rhythm shows up in hash-table load design, in estimating UUID collision probabilities in distributed systems, and in the sampling analysis behind data deduplication.

Question to ponder
To make it more likely than not that someone shares a birthday with you specifically, how many people must the room hold? The answer is 253—right back to that pair count. The same 253 is once "the number of pairs" and once "the number of people." Can you say clearly why these two 253s look identical?

Simpson's Paradox

Winning every layer, losing the whole
Causal Inference
Intuition

Treatment A beats B among mild cases, and beats B among severe cases too—A wins in every group. Yet merge the two groups and B's overall cure rate comes out higher. This is not an arithmetic slip; it recurs in real data (the famous 1973 Berkeley graduate-admissions "bias against women" was exactly this).

The trick hides in "who ended up in which group." Suppose A was mostly assigned the severe cases (hard to cure to begin with) and B mostly the mild ones—then A's overall score is dragged down by the difficulty it shouldered. The group proportions, not the efficacy, drive the merged numbers.

case severity → cure rate → A B per layer: A above B merged trend flips ⤴
Formal definition

At its core is an elementary but lethal inequality: even if $\dfrac{a_1}{b_1}>\dfrac{c_1}{d_1}$ and $\dfrac{a_2}{b_2}>\dfrac{c_2}{d_2}$, it does not follow that $\dfrac{a_1+a_2}{b_1+b_2}>\dfrac{c_1+c_2}{d_1+d_2}$. Fractions cannot be added term by term. Geometrically, two fractions are the slopes of two vectors in the plane; "merging" is vector addition, and the slope of the sum is reweighted by the lengths (sample sizes) of the components.

Why it's beautiful

It is the sharpest geometric demonstration that "correlation is not causation." The data does not lie; the question "which layer should I look at?" simply has no purely statistical answer—it must be settled by causal structure. Whether to split depends on whether the grouping variable is a confounder (split on it—like severity) or a mediator (do not split—it would block the real effect). Judea Pearl uses it to show: from the same numbers, different causal graphs yield opposite correct conclusions. Statistics was forced, for the first time, to admit that data alone is not enough.

Applications

It is the number-one trap in A/B testing and observational studies: the overall metric may rise while every user segment falls (only because the traffic mix shifted). In ML fairness, a model can be "calibrated" within every subgroup yet systematically biased in aggregate. Clinical trials, economic policy evaluation, and the segment analysis behind recommender systems must all draw the causal graph first and only then decide the aggregation level—otherwise they will confidently reach the opposite of the truth.

Question to ponder
Handed a table where "the grouped data says A is better, the merged data says B is better," which conclusion do you report to your boss? Before answering, ask yourself: if patients had been randomly assigned to A/B, would the answer change? (It would. Randomization severs the "who goes into which group" bias, and then you should trust the merged number—which makes exactly the point: whether you may aggregate is a causal question, not a statistical one.)

Bertrand's Paradox

The word "random" isn't finished being defined
Measure Theory
Intuition

Draw a chord at random in a circle. What is the probability it is longer than the side of the inscribed equilateral triangle? It sounds like a question with a single answer. Yet three equally reasonable ways to "draw at random" give three different numbers: $\tfrac12$, $\tfrac13$, $\tfrac14$.

Nobody miscalculated. The trouble is that "draw a chord at random" is never actually pinned down. "Random endpoints," "a random midpoint," "a random direction plus a random distance from the center"—all deserve the word "random," yet each sprinkles a different uniform distribution over the space of chords. Change how you sprinkle the points and the answer changes.

random endpoints → 1/3 random midpoint → 1/4 random radius point → 1/2
Formal definition

The three parameterizations correspond to three different probability measures. Using "the chord's midpoint" as the coordinate: the random-endpoints method makes midpoint density thinner near the center, giving $1/3$; the random-midpoint method spreads midpoints uniformly by area over the whole disk, giving $1/4$; the random-radius method makes midpoints uniform along a radius, giving $1/2$. The event "the chord is longer" simply has those three measures under those three measures. Without specifying a measure, the word "probability" has no definition.

Why it's beautiful

It slices straight through the soft spot of the "principle of indifference"—"with no reason to favor any outcome, assume they are equally likely": "equally likely" in a continuous space depends on which variable you make uniform. Uniform in the endpoints is not uniform in the midpoint. This is precisely why measure theory (Day 21) must serve as probability's foundation: probability does not grow automatically out of a "sample space"; you must additionally impose a measure, and it is a separate thing from the space itself. With one small geometry puzzle, Bertrand drags that deep necessity into the open.

Applications

It is the hidden reef of every Monte Carlo simulation: which variable your "uniform sampling" is uniform over directly decides whether the result is right—especially deadly on spheres and on parameter manifolds. The choice of an "uninformative prior" in Bayesian statistics is the same ghost: a uniform prior stops being uniform under a change of parameters, which is why we have Jeffreys priors and maximum-entropy priors that chase some invariance. E. T. Jaynes argued that adding the physical constraint "the answer should be invariant under translation and scaling" makes Bertrand's problem converge to the unique answer $1/2$—pinning down "random" by appealing to symmetry.

Question to ponder
If someone actually tosses a stick onto a circle drawn on the ground, which number does the measured frequency land on? (Most likely near $1/2$—because "tossing onto the circle" implicitly carries Jaynes's kind of translation-invariant physical sampling.) Then push further: does every "objective probability" secretly presuppose a measure we never stated out loud?

Deeper Reflection

Open Questions · Pushing to the boundary of the ideas
Are these four paradoxes four faces of the same mistake?
One thread ties them together: all arise from treating a hidden condition as if it weren't there. Monty Hall drops "the host knows"; the birthday problem drops "we're comparing pairs, not people"; Simpson drops "the grouping is a confounder"; Bertrand drops "the measure is not yet specified." In other words, the favorite blunder of probability intuition is mistaking a conditional probability for an unconditional one. Grant that, and the four paradoxes turn from four oddities into four exercises from the same lesson.
Why is the human mind so defenseless against quadratic growth (the birthday problem)?
Evolution gave us linear-extrapolation intuition: one more person, one more unit of risk—addition. But combinatorial explosion is multiplicative/quadratic, beyond the range of everyday experience. The same blind spot afflicts compound interest, exponential contagion, and network effects—anywhere the growing quantity is the number of relations rather than the number of elements, intuition underestimates. It explains why these domains most need "compute first, believe second."
Simpson says "let the causal graph decide the aggregation"—but where does the graph itself come from?
This is the real difficulty of causal inference. A causal graph cannot spring out of observational data (data yields only correlation); it comes from domain knowledge, interventional experiments, or extra assumptions. That is the core of Pearl's "ladder of causation": looking at data alone stays on the first rung, and answering "should I split?" requires climbing to the "intervention" and "counterfactual" rungs. The very existence of the paradox is evidence—pure data cannot fix causation; you must inject structure.
If "random" always requires choosing a measure first, is there any "objective probability" at all?
This is the watershed of the philosophy of probability. The frequentist anchors probability in "long-run frequency," seemingly objective—yet Bertrand shows even "how to repeat the experiment" must be agreed upon first, and objectivity slips away again. The Bayesian simply concedes that probability measures belief and depends on a prior. Jaynes's maximum entropy tries to force the prior to be unique via symmetry, keeping "objective" alive. There is still no consensus—but all four paradoxes point to the same humble sentence: probability is never a property of the world, but a composite of "the world + how we ask about it."