物理 · Physics

The Physics of Information

Day 9 · 2026 · Phase B Heat, Entropy & Statistical Mechanics
A bit is not an abstract 0 or 1 — it takes up space, costs energy, gives off heat. Information isn't a ghost hovering above the world; it is thoroughly physical.
A century and a half ago, Maxwell imagined a tiny "demon" stationed at a door between two gas chambers. By letting only fast molecules through one way and slow ones the other, it could split a glass of lukewarm water into a hot half and a cold half — doing no work, yet decreasing entropy. That stomps straight on the face of the Second Law. Physics took a full century to pin down the demon's flaw, and the flaw's name turned out to be information. The answer runs shockingly deep: to sort, the demon must know whether each molecule is fast or slow — it must measure and remember. But memory is finite; sooner or later it must be erased, and erasing one bit necessarily dumps at least one packet of heat into the environment. Here information and thermodynamics fuse into one, and for the first time "computation" gets an energy floor no one can sneak under — a floor that lies at the very bottom of today's AI energy problem.

Maxwell's Demon: Can Information Buy Free Work? Maxwell's Demon

Maxwell · 1867
Intuition Picture a box split into a left room and a right room, with a small door between them and a sharp-eyed, quick-handed "demon" sitting at it. The gas molecules come fast and slow (fast = hot, slow = cold). The demon watches the incoming molecules: see a fast one heading in from the left, open the door and let it pass to the right; see a slow one heading in from the right, let it pass to the left. Opening and closing the door costs essentially nothing. Gradually the right side fills with fast molecules (heats up) and the left with slow ones (cools down) — a uniformly tepid gas has sorted itself into a hot half and a cold half. But this conjures a temperature difference out of no temperature difference — it makes entropy decrease. You could even run a heat engine off that difference: a perpetual-motion machine that burns no fuel. Where's the error?
Mechanism Maxwell posed this thought experiment in 1867 precisely to argue that the Second Law is no ironclad mechanical theorem but a statistical regularity — true for vast numbers of molecules, not necessarily for a cheating operation on single ones. In 1929 Szilard boiled the demon down to its skeleton: imagine just one molecule in the box, and the demon need only know whether it is in the left half or the right half — that is one bit of information. On that single bit, the demon inserts a piston, lets the molecule push it, and extracts a definite amount of work from the surrounding heat bath:
W = kT ln2
W is the extracted work; k is Boltzmann's constant (the "exchange rate" converting temperature into energy, ≈ 1.38×10⁻²³ J/K); T is the temperature; ln2 is the natural log of 2 (≈ 0.69), coming from the two possibilities "left/right." This line nails, for the first time, one bit of information to a precise amount of energy: knowing where the molecule is is worth kT ln2 of work.
The counterintuitive part Many assume the demon's flaw is that "opening the door costs effort." It isn't. The door can be made arbitrarily light and opened arbitrarily cheaply — that road doesn't trap the demon. The real flaw hides in an act almost nobody notices: to sort, the demon must first know whether the molecule is fast or slow, left or right. It must measure — and remember. The demon isn't a pure mechanical device; it is a device that acquires information and has a memory. The bill, in the end, is charged to that memory — not to the door.
In a sentence: the demon isn't stealing work, it's stealing information — the question is, is information really free?
Ponder: If the demon's memory were infinite, could it cheat forever? What sleight of hand is hidden inside the assumption of "infinite memory"?
The catch: “infinite memory” doesn't exist physically, and not-erasing only defers the bill. Every measurement writes information into memory; finite memory eventually fills and must be erased, and erasure costs kT ln2 of entropy by Landauer — the books balance and the second law holds. “Infinite memory” secretly assumes a storage that can absorb infinite entropy, so the cheating premise itself fails.

Landauer's Principle: the Minimum Price of Erasing a Bit Landauer's Principle

Landauer · 1961
Intuition Sort logical operations into two kinds. Some are reversible: flipping a bit (0↔1), say — afterward you can always deduce the input from the output, no information lost. Others are irreversible — the archetype being erasure: whatever the bit started as, 0 or 1, after erasure it is uniformly 0. Two distinct inputs (0 and 1) get squeezed into a single output (0) — run it backward and you can't tell which it was, one bit of information is lost forever. Landauer's 1961 insight was: it is exactly this "loss of information" that forces a physical price.
Mechanism Picture the bit living in a "double well": left well = 0, right well = 1, a barrier between them. Erasing means driving the ball into the 0 well no matter which well it started in. Two possibilities (left or right) collapse into one — this storage subsystem's number of accessible states is halved, its entropy drops by k ln2. But the Second Law forbids total entropy from dropping: that expelled entropy must be dumped into the environment as heat. Hence the Landauer limit — erasing one bit releases at least:
Q ≥ kT ln2
Q is the heat that must be dissipated. At room temperature (T≈300 K) this comes to ≈ 2.9×10⁻²¹ J — about 0.018 electron-volts, astonishingly small, yet strictly greater than 0 (the number zero). The point is that inequality sign: this is not "current technology can't do better," but a floor nailed down by physical principle — no cleverness can ever erase more cheaply.
Notice the price hangs on erasure (logical irreversibility), not on "computation" itself. Reversible operations can, in principle, cost no energy at all — and that is exactly the key the next card uses to sentence the demon to death.
Before: 0 or 1 (2 possibilities) 0 1 erase After: always 0 (1 possibility) 0 ↯ heat released ≥ kT ln2 entropy kln2 is expelled, becoming heat
Erasure squeezes two possible inputs into one output — this step of losing information forces the system to dump at least kT ln2 of heat into the environment. It's a physical floor no computer can escape.
The counterintuitive part A purely logical act ("clear the variable") turns out to carry a thermodynamic price — and that price is independent of what material or circuit you use. Transistors, gears, DNA, neurons: whatever does a "many-to-one" erasure must pay that kT ln2. This is the sharpest cut of "information is physical": an abstract logical operation gets a price tag stamped on it by the laws of physics. Information is no longer a ghost floating above the hardware — it has a mass-like reality: it takes up space, spends energy, obeys thermodynamics.
In a sentence: computing is free, forgetting is not — erase a bit and you forfeit at least kT ln2 of heat.
Ponder: Why can "flip" (0↔1) be free while "clear" (all to 0) charges a fee? What fundamentally different thing does each do to the information?
Flipping (0↔1) is a one-to-one map (reversible): no information is lost, you can always invert it, so zero dissipation in principle. Zeroing is many-to-one (irreversible): it crushes both 0 and 1 to 0, erasing “which it was,” reducing logical information — and information can't just vanish, so it leaves as at least kT ln2 of heat. The charge isn't for changing state, it's for irreversibly losing information.

Death of the Demon: the Bill Was on the Memory Exorcising the Demon

Bennett · 1982 · information is physical
Intuition Now go back and settle with Maxwell's demon. Using one bit of information (molecule left or right), the demon extracted kT ln2 of work — on paper, a free lunch. But don't forget: that bit is now stored in the demon's head. To handle the next molecule, it must free up memory — it must erase the previous result. And erasure, by Landauer's principle, costs exactly kT ln2 of heat. What it earned and what it pays cancel to the last drop. After all that fuss, the demon's net gain is 0 (the number zero). The Second Law wasn't robbed of a cent.
Mechanism This settlement was delivered by Charles Bennett in 1982, and the chain runs like this: measurement itself can be done reversibly, dissipating no energy (Landauer and Bennett showed reversible computation costs zero energy in principle); extracting the work is fine too. The one unavoidable dissipation is the step where the demon must clear its memory in order to work in a cycle. In other words, the old belief that the cost lay in "measurement" was wrong — what truly charges a fee is forgetting.
earn: kT ln2   pay (erase memory): kT ln2   ⟹   net = 0
Every time the demon extracts a packet of work, it takes on a debt — "a bit waiting to be erased"; when it finally erases that bit to complete the cycle, the debt is repaid in full. The Second Law isn't violated but rescued by a hidden ledger entry called information.
Why it matters This century-long detective story has a one-line verdict: information is physical (Landauer's dictum). A bit is not a mathematical abstraction; it must be carried on the state of some physical system — occupying space, carrying energy, taking part in the entropy ledger. The demon can be subdued precisely because we finally admit: the "knowledge" in the demon's head and the "heat" in the gas chambers are two entries in the same thermodynamic ledger, interconvertible and settled together. The entropy-and-arrow-of-time installment cast entropy as "disorder"; here it shows another face — entropy is the information you lack; two faces, the same k ln.
Cross-disciplinary reading · biology / engineering / theory of computation This exchange rate between information and energy traps far more than one demon in a thought experiment:
  • Biology: molecular motors inside cells (like kinesin "walking" along microtubules), ion pumps, even the error-correcting copying of DNA have been argued to be real Maxwell demons — they rectify Brownian motion by reading molecular information, and the cost is still spent free energy (hydrolyzing ATP), not a cent less.
  • Engineering: reversible computing is a real field — using logic gates that don't erase information (like the Toffoli gate), one can in principle push per-step energy below the Landauer limit, pointing the way to ultra-low-power chips (quantum computing is inherently reversible and lives by the same logic).
  • Theory of computation: run "information = physical entropy" the other way and you get the whole viewpoint of analyzing algorithmic energy cost with statistical mechanics, of seeing learning as "compressing information" — entropy, information, and energy as three faces of one thing.
Wherever you use information to tame randomness, Landauer's tollbooth is standing there.
In a sentence: the demon didn't lose at measuring, it lost at forgetting — the instant it clears its memory, it repays the stolen work in full.
Ponder: If a device only records and never erases, could it truly violate the Second Law? And how long could it last? (Hint: is there enough storage in the universe for it to run forever?)
Short-term it looks like a violation; long-term no. Recording-without-erasing just parks entropy in the memory instead of dumping it — the total ledger isn't dodged, only filed under “memory.” It lasts until the storage runs out — and the universe's usable storage (matter, energy) is finite, so it fills eventually and must either halt or pay to erase. The second law isn't broken, just postponed.

The Thermodynamic Floor of Computation: AI's Ultimate Energy Limit The Thermodynamic Limit of Computation

Landauer limit · physics of computation
Intuition Landauer's principle doesn't just try the demon; it draws an energy floor under every computer: each erased bit must dissipate at least kT ln2 of heat. How far are today's chips from that floor? Terrifyingly far. A single transistor switch actually spends roughly ten thousand to a hundred thousand times the Landauer limit. That means there is physically four to five orders of magnitude of room left to save energy — but it also means that however far technology advances, this floor can never be pierced: a machine erasing a billion-billion bits per second is destined, just by "forgetting," to give off a heat that cannot go lower.
Mechanism Translate the floor into intuition: at room temperature, erasing one bit ≈ 2.9×10⁻²¹ J. It sounds negligible, but multiply it by the operations per second in today's data centers and the Landauer limit becomes a theoretical minimum power you can hold up against a real electricity meter. In the real world, training a large model burns electricity measured in megawatt-hours, the vast majority spent not on this physical floor but on engineering losses far above it (leakage currents, drive, cooling, moving data around).
This yields a sober two-part conclusion. First, AI's energy bottleneck is not a law of physics right now, but engineering — there's still a ten-thousand-fold margin to the Landauer floor, an enormous room to economize. Second, the floor really does exist: reversible computing can approach it but not pass through it, unless the computation never erases information (which then demands unbounded storage). Physics gives both hope and a hard limit.
The counterintuitive part The human brain runs on about 20 watts and finishes cognition that today's supercomputers can only chase, with an energy efficiency many orders of magnitude above top-end chips — it sits much closer to the Landauer floor. This hints that low-power intelligence is entirely allowed by physics; we just haven't learned how to build it. When someone says "AI's power draw is an insurmountable wall," physics answers calmly: the truly insurmountable wall (the Landauer floor) is still far away; what we've hit is merely the temporary ceiling of our own fabrication. Making more of our computation reversible and erasure-light is the honest road toward the physical limit.
In a sentence: AI's energy ceiling isn't a law of physics but engineering — the real physical floor lies ten thousand times below.
Ponder: If a computer could make most of its operations "reversible, erasure-free," could it think at nearly zero energy? Where would that machine's cost get shifted to instead?
In principle reversible computing can push per-step dissipation far below kT ln2, near zero energy. But the cost shifts elsewhere: reversibility means keeping all intermediate information → storage and volume explode; reading out the useful answer and discarding junk bits ultimately requires erasure, paying Landauer's bill; and it must run very slowly (low dissipation needs near-equilibrium). You save energy at the price of time, space, and one unavoidable final settling of the books.

Going Deeper

Is entropy "objective disorder" or "the information I lack"? This installment slammed them together.
Two seemingly different entropies are proven, here at Landauer, to be the same quantity. Thermodynamic entropy S=k lnW counts microstates; Shannon information entropy counts "the bits you don't know." Settling Maxwell's demon works only if you accept "erasing one bit = losing one packet of physical entropy k ln2" — which nails the two into two sides of one coin, the exchange rate being exactly Boltzmann's constant k. The statistical-mechanics installment already planted this line with Jaynes's maximum-entropy view: entropy measures "your most honest ignorance given the macroscopic constraints." Here it fully ignites — disorder and ignorance are the same thing.
Can reversible computing really cost zero energy? Then why is my laptop still hot?
In principle, as long as you never erase information, computation owes no thermodynamic debt — using reversible logic gates like Toffoli or Fredkin (input and output in one-to-one correspondence, always run-backward-able), per-step energy can approach 0 without limit. Your laptop is hot because it is erasing constantly: every register overwrite, cache clear, discarded intermediate result is a "many-to-one" operation Landauer charges for — and actual energy runs ten thousand times above the floor anyway (leakage, drive losses). Reversible computing's cost lands elsewhere: either keep all the intermediate garbage (eats storage) or "uncompute" to clear it reversibly (eats time). Energy, space, time — the three are traded against each other; there is no true free lunch.
Why exactly ln2, and not some other number? Where does the "2" come from?
That 2 is simply the two possibilities of one bit: 0 or 1, left or right. Erasure squeezes 2 equally likely states into 1, dropping the accessible-state count from W=2 to W=1, an entropy change of ΔS = k ln1 − k ln2 = −k ln2. If you erase a storage cell holding d states (say an octal digit, d=8), the cost is kT lnd. Inside the ln is always "the number of possibilities you annihilated." This also explains why information is measured in log: turning a count of possibilities into additive bits requires a logarithm — the same log runs from entropy S=k lnW straight through to Shannon information, one shared skeleton.
Has anyone actually "seen" the Landauer limit in a lab, or is it pure theory?
Seen. In 2012 a French group used optical tweezers to trap a single colloidal particle in a tunable "double well," treating it as one physical bit, and measured that erasing it dissipates heat approaching kT ln2, beautifully confirming Landauer's principle (Nature, 2012). It has since been verified in nanomagnets, single-electron devices, and even quantum systems. This 1961 principle walked off the page into measurable experimental fact — "information is physical" is no longer a slogan but a number you can read off a thermometer.
How much information can the universe itself store, how many operations can it run? Is there an ultimate limit on information?
There is, and it runs frighteningly deep. Put Landauer, quantum mechanics, and gravity together and you get a startling limit: the maximum information a region can hold is proportional to its surface area (not its volume!) — the seed of the Bekenstein bound and the holographic principle, with a black hole's entropy exactly equal to its horizon area (in Planck-area units) divided by 4. Cram in too much information and the object collapses into a black hole. Seth Lloyd has even estimated how many operations "the universe as a computer" could have run so far (about 10¹²⁰). These threads lift information to a footing as fundamental as space, time, and gravity — and the information paradox of the black-holes-and-gravitational-waves installment catches fire right here.

Further Reading