Most of the neurons in your head are doing absolutely nothing right now — and that idleness is the entire reason the organ gets away with twenty watts.
Training a large model burns through a small town's worth of electricity. Meanwhile you are reading this sentence and simultaneously hearing someone talk downstairs, on twenty watts — less than a light bulb. A gap that wide is usually not something you close by optimising harder; it means the two machines were never the same kind of thing to begin with. This one is about a group of engineers who set out to copy the brain's homework in earnest — not its knowledge, but its temperament: sit dead still until something actually happens, then say one word. That road has produced real chips and a real camera. It has also run into a fairly hard wall.
Start with what a "neuron" means inside today's AI. It is a small piece of arithmetic: take the numbers arriving from upstream, multiply each by a weight, add them up, push the sum through a simple transformation, emit one number — say 0.73. A few thousand such units finish at once and hand a few thousand numbers to the next layer, and the whole network grinds downward that way. Note one detail: whether or not the input changed, every unit computes and emits something on every single round.
A real neuron does none of this. It emits no number at all; it merely goes "pop" at some moment — a few-millisecond electrical spike of fixed size, present or absent, with no such thing as "firing 0.73 of a spike" (Topic 35 covered this all-or-none alphabet, Neuron & action potential). So what it sends downstream is not a value but a scatter of points along time: when it fired, how long since the last one, whether it fired together with the cell next door. The information lives entirely in the timing of those points.
The standard way to write that down as something you can actually run is called leaky integrate-and-fire. The name is forbidding; the thing is a cup with a small hole in the bottom. Every incoming spike drips a little water in, while the cup leaks continuously. When the water reaches a mark on the wall, the whole cup tips over — that tipping is its one spike — and refills from empty.
The magic is in that cup's default state: with no input, it does nothing at all — no computation, no output, no power drawn. This is what event-driven means: the network does not tick along on a clock, it is pushed forward only by things that actually happen. Of the 86 billion or so neurons in your head, the vast majority are silent this second.
So what does that 0.73 an artificial neuron emits correspond to, biologically? The firing rate — how many spikes per unit time, denser meaning 0.9 and sparser 0.1. Which is to say: today's deep networks have flattened the neuron's time dimension into an average. That simplification bought something enormous (a whole layer becomes one matrix multiplication, exactly what a GPU likes) at the cost of throwing away the entire "when" axis — and a certain amount of the brain's computation lives precisely there (the phase codes of Topic 38, the temporal codes of Topic 35). Spiking networks are the attempt to pick time back up.
Why should "shut up when nothing happens" be worth orders of magnitude? Because of a counterintuitive fact about modern chips: the electricity mostly does not go into computing. It goes into moving data around.
Horowitz laid the accounts out for the industry in 2014. At the process node he measured, a 32-bit floating-point addition cost about 0.9 picojoules (a picojoule is a very small unit of energy; the absolute number matters less than the comparison coming next), while fetching a single number out of main memory cost several hundred — a gap of roughly 700×. So when a GPU runs a model, most of the power is burned hauling hundreds of billions of weights from memory to the arithmetic units. Computing is nearly free. The round trip is the bill.
This has a root cause: today's computers keep the place that stores things and the place that computes on them physically apart, with a channel ferrying data between. The brain does not have this problem, because its "weights" are the synapses themselves, and the computing happens right there at the synapse. A signal arrives, the synapse passes it on scaled by its own strength, and the result goes straight into the next cell. Nothing is fetched, moved, or written back.
The second saving is sparsity, and it too is not an elegance but a consequence of the electricity bill. Lennie ran the numbers carefully in 2003: one cortical spike costs roughly 2.4 billion molecules of ATP (the cell's universal energy currency). Divide the brain's total power budget by that unit price and you get — fewer than about 1% of neurons can be substantially active at any moment. Your brain did not "choose" sparsity. Its budget can only afford sparsity.
Once you know the expensive part is the fetching, a pile of apparently unrelated engineering tricks snaps into a single line. Quantisation (squeezing weights from 32 bits to 8 or 4) saves not arithmetic but bytes moved per fetch; the KV cache saves re-fetching things already computed; mixture-of-experts (waking only a small slice of the parameters each time) saves hauling the weights this token doesn't need. All of them are doing one thing: fetch less. Which is precisely the problem the brain is structurally immune to.
Carving that temperament into silicon is a field called neuromorphic engineering — Carver Mead's coinage, 1990. His proposal was sharper than mere imitation: don't use transistors to simulate equations, let the transistor's own physics do the computing. He had noticed that the way current varies with voltage in a transistor held at very low voltage has a shape strikingly like the behaviour of ions crossing channels in a cell membrane. So take the short cut and let physics do that step directly.
TrueNorth (IBM, published in Science in 2014) was the first result that genuinely made people sit up: a million neurons, 256 million synapses, 5.4 billion transistors, running in real time on 70 milliwatts — a power density roughly four orders of magnitude below a conventional processor. The catch: it could only run inference. It could not learn.
Loihi (Intel, 2018) closed that gap: 128 cores, about 130,000 neurons, 130 million synapses, and learning on the chip itself — the plasticity rule is programmable, so local rules of the STDP family ("who fired first decides whether the connection strengthens or weakens") run directly on silicon (Synaptic plasticity). By 2024 Intel had packed 1,152 second-generation Loihi chips into a chassis the size of a microwave, called Hala Point: 1.15 billion neurons at up to 2,600 watts — a neuron count in the neighbourhood of an owl's brain.
SpiNNaker (University of Manchester, led by Steve Furber, one of the designers of the original ARM chip) took a third route entirely: no special neuron circuits, just a million ordinary little ARM cores, with all the ingenuity poured into a routing network built for spike traffic — because spikes are very short, very numerous and scattered to very many destinations, and the hard part was never the computing but the delivery.
The idea that is actually out doing work, though, is a camera. The event camera (the founding chip came out of Zurich in 2008, from Lichtsteiner and Delbrück) applies "shut up when nothing happens" to imaging: there is no such thing as a frame. Each pixel minds its own business and reports only when the brightness at its own point changes — reporting "I got brighter" or "I got darker" with a microsecond timestamp. Point it at a static scene and its output is zero. The idea is lifted from the retina, which never sent the brain a series of photographs either, but mostly a report of what changed (Visual pathway).
And the payoff is not only power. With no frame to wait for, latency drops from milliseconds to microseconds. Because each pixel handles its own light level independently, sunlight and deep shadow in the same scene are both legible (that capacity is called dynamic range; it reaches 120 dB here, against roughly 60 for a normal camera). And fast motion does not blur — blur is a by-product of averaging over an exposure, and this sensor never exposes.
Now the honest part. Wall one: these networks are very hard to train. Every AI that learns today runs on backpropagation (Topic 39), which has a prerequisite — each component must be able to answer "if I changed slightly, how much would the final error change?" But "spike or don't spike" is a cliff: a hair below threshold and nothing happens; a hair above and the whole spike fires. That jump has essentially no slope anywhere, so the credit coming back is all zeros and nothing learns.
The standard workaround is called the surrogate gradient, and it is faintly disreputable but effective: on the way forward the neuron spikes for real, cliff and all; on the way back, pretend the cliff is a gentle smooth slope and use that fake gradient to assign blame. There is no rigorous guarantee behind it, but it trains usable networks. The other route is to train a conventional network first and convert it into a spiking one. Both are a little ironic: the low-power chip still has to be taught by power-hungry GPUs.
Wall two: the advantage is extremely picky about the job. Intel's own 2021 survey is admirably frank: the wins cluster in work that is sparse to begin with and needs an answer immediately — keyword spotting (always listening for you to call it, where the point is that 99.9% of the time nothing happens), closed-loop robotic control, certain optimisation problems, and odour recognition (Imam and Cleland built a circuit modelled on the olfactory bulb on Loihi in 2020; it learns a smell in one exposure and still recognises it under heavy interference). But the moment the task is "push one large batch of data through a dense network", GPU pipelines win crushingly — because in that regime there is no sparsity to exploit in the first place.
Wall three is the most mundane and the most lethal: no ecosystem. Half the reason deep learning took off is a toolchain anyone can pick up in an afternoon. Over here, every chip has its own way of being programmed, the library of runnable models is thin, and so few people bother to learn it.
Which is why treating it as "a better GPU" is a category error. It is a different kind of machine, and it is a bet on something not yet settled: whether what constrains us next is being too slow, or drawing too much power and reacting too late. If it's the former, it has little future. If it's the latter — the things on your wrist, in your ear, flying overhead, that must stay awake permanently and cannot be plugged in — it is the only direction that works.
This field also adds an unexpected footnote to the argument in Topic 39. Back then, "a synapse can only sense its own two ends and cannot reach global information" was filed as a biological defect. On silicon it turns out to be the most valuable property in the room: a local rule never has to haul the error across the whole network, so a chip can learn while it works, without shipping data back to a centre and waiting for a table of corrections to come down. That is exactly what lets Loihi put plasticity on-chip. Biology's compromise, on a different substrate, becomes engineering's advantage.
"If it hasn't changed, don't speak" looks like a power-saving trick, but several very different disciplines have arrived at it from their own directions: