Before any definitions, just stand in them.
The first is a laboratory. A cup of warm water sits on the bench; heat flows from water to air and never the other way. Yet write out Newton's equations, Maxwell's equations, Schrödinger's equation, and not one of them distinguishes a direction of time — swap t for −t and they all still hold. Cups shatter; they never reassemble. In all of physics, exactly one law knows which way time points.
The second is Bell Labs in 1948. The mathematician Claude Shannon is asking a purely engineering question: how few 0s and 1s does it take to get a message down a wire intact? He writes down a formula for how uncertain something is, and finds he has written the same formula the physicist Ludwig Boltzmann had carved on his tombstone seventy years earlier.
The third is a GPU mid-training run. A number on screen called the loss falls from 10.8 to 2.3. It isn't a dimensionless progress bar: it measures how much information, on average, it costs to encode real text using the model's picture of the world. Entropy here is no longer something observed. It is the thing being optimised.
The fourth is a cell. It assembles scattered atoms into membranes, proteins, DNA — order rising against the current, apparently in defiance of the second law of thermodynamics, the one saying disorder only ever increases and never decreases on its own. It isn't. It simply dumps a larger quantity of disorder into its surroundings, and the books balance at the wider scale.
The fifth is somebody's mind. They have just come back from a week of doing nothing and feel worse than they do at work: unresolved worries flood in and fight with their goals, and their attention is churned into a mess. Psychologists named that state psychic entropy.
Put the five side by side and the easy question is whether they're related. But that question is too shallow. The genuinely strange thing is this: molecules, letters, words and cells share no physical substrate whatsoever, so why does one piece of mathematics govern all four?
Answering that means first getting rid of the slogan "entropy equals disorder." It's handy, but it turns entropy into an adjective, when entropy is really an act of counting.
S = k log WBoltzmann's 1877 formula was later carved on his headstone. S is entropy, k is Boltzmann's constant (the exchange rate that turns a raw count into physical entropy units), and W is the number of microstates — how many molecule-level arrangements are fully compatible with the macroscopic appearance you actually observe: same temperature, same pressure, same colour. A neatly ordered deck of cards admits only a handful of arrangements; the messy-looking ones number astronomically. You will essentially never shuffle your way back to order, not because messiness has any special pull, but because the messy arrangements overwhelmingly outnumber the tidy ones.
Two things here matter later; the rest is on the source pages.
First, the second law is not an iron rule but a probabilistic one. It never says entropy cannot decrease, only that a decrease is absurdly improbable — warm water separating spontaneously into a hot half and a cold half violates no law of mechanics, but for a cup of gas holding some 10²³ molecules, waiting for it would take far longer than the universe has existed. What makes the law hard is precisely that magnitude: at 10²³, a statement about probability becomes, in practice, iron. That hardness has a measure you can compute — relative fluctuation falls off as 1/√N. At a hundred molecules the energy wobbles by around ten percent; at 10²³ it is smaller than any instrument can register. Shrink N and the fluctuations come straight back as the main character: nanomachines and the molecular motors inside a cell live in a sea of them. None of thermodynamics' iron laws are iron. Large numbers just press them flat enough that the seams stop showing.
Second, entropy is not an intrinsic property of an object. Take one cup of water: care only about temperature and volume and its entropy is enormous; record the position and velocity of every molecule and it occupies exactly one microstate, entropy zero. So computing any entropy means first declaring which macroscopic variables you're bundling microstates under — that choice of how coarsely to look is called coarse-graining, and it's step one of writing the formula, not philosophy. (The physicist Carlo Rovelli pushes this to its limit in The Order of Time, arguing that even the difference between past and future comes from that blurring — a very attractive conjecture, and one the field is far from agreeing on.)
H = −Σ p(x) log₂ p(x)Shannon wrote this down in 1948 while working on communication. It measures the average surprise of a probability distribution, in bits — one bit being the information needed to halve the space of possibilities. Entropy here is two things at once: before the reveal it's uncertainty, after the reveal it's information.
What matters is how he got there. He wasn't copying Boltzmann; he derived it from three axioms — continuity, monotonic increase as options multiply, and decomposability — and proved that −Σ p log p is the only form satisfying all three.
Now set the two formulas side by side. When W microstates are equally likely, Boltzmann's S = k log W and Shannon's H differ only by the constant k. One man was counting molecular arrangements, the other measuring the uncertainty of a message, and two entirely independent routes arrived at the same place. The mathematician John von Neumann is said to have advised Shannon to keep the name "entropy" on the grounds that nobody really knows what entropy is, so he'd always have the advantage in an argument.
And the number went to work immediately: the source coding theorem proves that no lossless code can average fewer than H bits. That's a floor nothing gets under, not a figure of speech.
The same paper carries something stranger still. Give a channel a capacity C, and Shannon proves that at any transmission rate below C a code exists that drives the error rate arbitrarily close to zero, while above C reliable transmission is impossible. That overturned the old belief that speed buys errors: driving errors to zero does not require driving the rate to zero, only staying under the line. The lovely part is that he proved such good codes must exist by picking codes at random — without saying what any of them looks like. Building one took humanity another half-century.
In the first two rooms entropy is observed and computed. Here it becomes the thing being minimised.
Training a language model means predicting the next token given the previous N: the model emits a probability distribution q over the whole vocabulary, and the true answer is one specific token. The gap is measured by cross-entropy:
H(p, q) = H(p) + KL(p‖q)Read it as: the average number of bits per symbol it costs to encode data that truly follows p using a code optimised for q. H(p) is the data's intrinsic entropy, independent of your model; KL is the penalty for using the wrong distribution. Since the first is constant, minimising cross-entropy means making the model's distribution approach the true one.
What matters is that these numbers have a scale. An untrained model on a 50k vocabulary sits at loss ≈ ln(50000) ≈ 10.8, exactly the uniform-guessing baseline; a well-trained LLM on web text runs about 2.0–2.5, a perplexity of 8 to 12 words. Raw UTF-8 text runs about 5 bits per byte, and a well-trained LLM compresses human text to 30–50% of its original size, comfortably beating gzip's roughly 70%. None of that is metaphor: cross-entropy is denominated in bits, which is why "compression is intelligence" can be argued as a proposition rather than repeated as a slogan.
The fourth room poses the oldest question. Life assembles scattered atoms into cells, order rising against the current — surely that violates the second law?
It doesn't, and the operative word is isolated. The second law governs the total entropy of isolated systems, and life is an open system: it takes in low-entropy energy and expels high-entropy waste, and so long as it exports more than it removes from itself, the books balance.
Those books can be made very concrete. A seated adult radiates roughly 100 watts of heat, about the same as an old incandescent bulb — the entropy tax you pay the universe to keep your insides orderly. Scale up to the planet: Earth receives a small number of high-energy, low-entropy visible photons from the Sun and radiates back roughly twenty times as many low-energy infrared photons. Energy in and out are nearly equal; entropy rises sharply. That spread funds every scrap of local order on Earth.
Dissipative structures — the chemist Ilya Prigogine's term for ordered forms that exist only while a flow of energy passes through them — invert an old intuition. Order doesn't mean equilibrium and stillness; push a system far from equilibrium and order emerges spontaneously. Heat a thin layer of oil in a pan and, once the temperature difference crosses a critical point, the whole layer organises itself into a tidy grid of hexagonal convection cells with no external design at all. The corollary matters later: structure is fundamentally a verb rather than a noun. A whirlpool, a candle flame, a cell, a company — each exists only while energy streams through it, and dissolves when the stream stops. (Distributed systems corroborate this nicely: the mechanism Cassandra and Dynamo use to hold replicas consistent is called anti-entropy repair.)
Back to the question at the end of §1: molecules, letters, tokens and cells share no physical substrate, so why does one piece of mathematics govern all four?
Because all four rooms are asking the same question: given what I can see, how many possibilities remain underneath?
The physicist can see temperature and volume but not where each molecule is. Shannon can see a probability distribution but not which outcome this draw will produce. The person training a language model can see the model's predicted distribution but not the next real token. The biologist can see the cell's macroscopic order but not its current molecular configuration. All four are in the same position: holding a coarse description, with a mass of indistinguishable detail underneath it.
And for "how many possibilities remain," the mathematics doesn't ask what those possibilities are made of. It only needs a probability distribution. Molecular positions, letters of the alphabet, tokens in a vocabulary, metabolic states — feed them into the same formula and the same kind of quantity comes out. So this isn't one metaphor getting lucky and landing on another; it's the same mathematical object being discovered independently in four places.
The strongest evidence that the connection is real rather than merely elegant is Landauer's principle: erasing one bit of information must dissipate at least kT ln2 of heat into the environment. That gives bits and joules an exchange rate. It has units, it's measurable, you can take it into a lab: at room temperature the figure is about 2.9×10⁻²¹ joules — startlingly small, strictly greater than zero, and independent of what the thing is built from. Transistors, gears, DNA, neurons: anything performing a many-to-one erasure pays it.
And it settled a question that had been open for a very long time. Maxwell's demon, proposed in 1867, is a thought experiment: a small demon who can see each molecule's speed need only open a valve for the fast ones and block the slow ones to create a temperature difference at no cost in work, and the second law breaks. The paradox held out for more than seventy years. The answer turned out to lie in information — and not quite where people had assumed. When the physicist Charles Bennett closed the account in 1982, he pointed out that measurement is not what costs: measurement can in principle be made reversible and free. Forgetting is what costs. The demon extracts kT ln2 of work using one bit, but that bit now sits in its memory; to handle the next molecule it must clear the memory, and erasure owes exactly kT ln2 by Landauer's principle. The gain and the debt match to the last decimal, and a full cycle nets the demon zero. The second law was never violated. It was rescued by a hidden ledger entry called information.
That is the test of whether a cross-domain connection is worth anything. Not that it feels elegant, but that it resolved a specific, long-standing, previously intractable problem.
Step back from entropy and something more general comes into view.
When a concept moves to a new field, sometimes only the word makes the trip and sometimes the reasoning behind it comes too. The difference is enormous. If only the word travels, you've gained nothing. If the reasoning travels, you inherit everything that has already been proved about it — theorems, lower bounds, impossibility results. None of it has to be proved again, and proving it the first time often took decades.
The exchange rate keeps paying out. Landauer's limit sets an energy floor under every computer, and a transistor switch today costs somewhere between ten thousand and a hundred thousand times that floor. Which means the claim that AI's power draw is an immovable physical wall can be answered directly: the immovable wall is still four or five orders of magnitude away, and what we have run into is a temporary ceiling of our own manufacturing. A human brain does cognition today's supercomputers cannot touch on about twenty watts, far closer to the floor. It takes a quantity with units to make that argument settleable at all.
There are more everyday returns too. Mutual information — how much knowing X reduces your uncertainty about Y — gives "is this metric worth watching" a computable test: a signal's value equals its mutual information with the outcome you actually care about. KPIs fail, which is Goodhart's law, precisely because optimising the proxy destroys the information it shared with the real goal. That isn't a motivational maxim; it's a quantity you can estimate.
So the point of connecting disciplines comes down to not paying twice for the same invention. When the same mathematical structure keeps turning up in different fields, a result you worked hard for in one of them can simply be collected in another. That is the whole case for doing it.
And precisely because what you collect is real, it's worth knowing where it stops travelling.
Back to the fifth room, and the whole neighbourhood around it: rooms get messy on their own, organisations suffer entropy, knowledge decays, relationships need work or entropy takes them.
Every one of those is intuitively true. The question isn't whether the conclusion is right but what's holding it up — what they borrow is the sense of inevitability the phrase carries: this is a law of physics, the universe is built this way. That inevitability was earned by the preceding sections, on the strength of countable microstates, a declared coarse-graining, a number with units, and the magnitude 10²³. The four questions below aren't for catching people out. They're for deciding whether a claim can carry weight.
One. What are the microstates, and which coarse-graining are you bundling them under? For a cup of water the microstate is every molecule's position and velocity and the macroscopic variables are temperature and volume — both specified, so W is countable and S computable. What is a microstate of a room? The position of every dust particle? Then the room's entropy soars regardless of whether you tidy. The location of each object? Then you must first define what counts as an object — a book counts, but does each page? Change the coarse-graining and the answer changes by arbitrary factors. If you can't say what you're counting, you can't write down W.
Two. Does the number have units, and can you report its value at two different times? You can say a text has an entropy of 4.7 bits per character, or that after three days of training the loss went from 2.5 to 2.3. But what was your organisation's entropy last quarter? Nobody can report that number, and nobody really wants to. A quantity that only ever appears in the comparative and never as a value is hard to call a quantity.
Three. Does the "law" emerge from large numbers? The second law is hard because of 10²³. How many "particles" are in a room? Dozens of objects. At that magnitude the overwhelming-majority argument has no numbers behind it. Rooms do get messy, but by a biased random walk — you take things out more often than you put them back. That's a behavioural explanation rather than a thermodynamic one, and it predicts something quite different: hire someone to tidy daily and the room stops getting messy. The defining feature of the real second law is that you cannot build any machine that beats it, not that things get worse when neglected.
Four. Are you treating an open system as an isolated one? The second law governs isolated systems, and every object in the social analogies — a team, a codebase, a marriage, a body of knowledge — is an open system. The accounting in §5 holds up precisely because it honestly tallied both directions: so much low entropy in, so much high entropy out. Skipping that ledger and keeping the conclusion skips the only difficult step.
The psychologist Mihaly Csikszentmihalyi says openly in Flow that he borrowed the word from physics: the default state of consciousness trends toward disorder, anxiety is entropy (challenge overwhelms skill) and boredom is entropy too (skill has nowhere to go), and flow is the opposite. Take the four questions in turn. Microstates? None. Units? You can't report that this afternoon's psychic entropy was 3.2. Large numbers? No. Open system treated as isolated? Partly.
But the conclusion isn't that psychic entropy is nonsense. It's a rather good naming — filing anxiety and boredom, two apparently opposite forms of suffering, under a single category as mismatches between attention and goals, which is a genuine conceptual unification. What it borrowed is entropy's shape: a default drift toward disorder, order requiring sustained investment. The problem was never the borrowing; it's failing to return — not admitting that what was borrowed is a shape, and then invoking the authority of a physical law to underwrite a psychological claim.
The dissipative-structure analogy stands up far better. Calling a codebase a dissipative structure explicitly treats it as an open system and demands you answer where the energy comes from and where the entropy goes. It's a structural analogy, and the structure itself can be checked: you can go and identify the flow, and see what happens when it stops. It still yields no number, which makes it a good model rather than a measurement. That distinction is worth defending.
This tier has debts of its own to acknowledge, though. Prigogine proved a minimum-entropy-production theorem early in his career, but it holds only near equilibrium, in the linear regime where flows stay proportional to forces — and dissipative structures all live outside that regime. So in the genuinely interesting territory there is no general principle saying a system optimises anything. Using minimum entropy production to explain life, organisations or cities carries a near-equilibrium theorem into a place it explicitly does not apply; and the principle pointing the other way, maximum entropy production, remains an open hypothesis rather than a law. A plainer caution too: "life is a dissipative structure" is true and thin, because so are hurricanes and candle flames.
Taking something apart without offering a replacement is just a different rhetoric. Roughly three tiers.
When it's literal mathematics, actually compute it. Whenever you have a probability distribution, entropy isn't a metaphor but a number you can put through a calculator. The mental models site offers a good practical test: to judge whether a piece of information has value, ask how much it shrank your uncertainty. If your internal probability distribution hasn't budged after reading a report, its information content was zero, however long it ran. A two-hour meeting that leaves your judgement unchanged contributes nothing either.
When it's a structural analogy, state the whole thing. Dissipative structures are the model case, and the condition for using them is naming all three parts: which flow sustains this order, where does the energy come from, and where does the entropy go? If you can't answer the third, you're not drawing an analogy, you're borrowing gravitas. There's a simple self-test: analogies at this tier can usually be restated without the word "entropy" at all, losing nothing. If it translates cleanly, what you borrowed was a causal story you can spell out. If nothing remains once you delete the word, the word was all it ever had.
When it supplies only inevitability, it's rhetoric. Look at what the word is doing in the sentence. If it explains — why this happens, what comes next, where to intervene — it's earning its place. If it excuses, as in "organisations bloat, that's entropy, nothing to be done," it's a shortcut, and what it leans on to take it is the authority of physics. The genuine second law is never used to excuse anything; it delivers relentlessly specific conclusions. A refrigerator can cool its interior, and the price is your electricity bill. The Carnot limit, which sets the ceiling on any heat engine's efficiency, gives every engine a target it can never reach but is worth approaching. Stop trying to build a perpetual motion machine. A real law tells you what the price is; it doesn't tell you to stop struggling.
Finally, back to those four rooms. The connection between them is real: physical and information entropy are one mathematical object, Landauer's principle makes bits and joules convertible, life's entropy ledger can be tallied down to the watt. That unity isn't a coincidence. It comes from a question plain enough to cross any disciplinary boundary — given what I can see, how many possibilities remain underneath?
Worth keeping an eye on the word, then — not out of fear that someone will misuse it, but because concepts that genuinely travel from one field to another, losing nothing on the way, are rare enough to be worth guarding.