Day 22 · 2026.07.14

Differential Equations

Writing the laws of a changing world in the language of rates
"Ever since Newton, humanity has come to realize that the laws of physics are always expressed in the language of differential equations." — Steven Strogatz

Ordinary Differential Equations

Using "the rate of change right now" to unfold an entire future
ODE
Intuition

A differential equation doesn't tell you "where you are." It tells you "at this position, how fast you are changing." It's a local rule; follow it one small step at a time and it grows an entire trajectory of destiny.

Take population. The crudest assumption: new births are proportional to the current count — more people, more births — so $\frac{dP}{dt}=rP$, whose solution is exponential blow-up. But Earth can't hold infinitely many people; as resources tighten, growth should brake. Verhulst added a "brake term": as $P$ nears the carrying capacity $K$, the growth rate is throttled to zero by itself. The equation becomes $\frac{dP}{dt}=rP\left(1-\frac{P}{K}\right)$, and the trajectory bends from an exploding exponential into an elegant S-shaped (logistic) curve: slow, then steep, then saturating. No one "designs" this S-curve — it is the shape that the local rule grows for itself.

Formal definition
$$\frac{dP}{dt}=rP\left(1-\frac{P}{K}\right)$$

$P(t)$ is the population over time, $r$ the intrinsic growth rate, $K$ the carrying capacity (the ceiling the environment can sustain). The bracket $1-P/K$ is the brake: near $P\approx 0$ it is about $1$ (near-exponential growth); as $P\to K$ it tends to $0$ (growth halts). Given a single initial value $P(0)$, the whole curve is uniquely determined.

K · carrying capacity exponential rP logistic S t P
Why it's beautiful

The same logistic equation describes, almost unchanged: bacterial growth, the spread of rumors and viruses, technology-adoption curves, tumor growth. These utterly unrelated phenomena share one skeleton — "self-limiting growth." This is what is most astonishing about differential equations: they don't describe one particular story, but capture the shared grammar of growth behind a whole class of stories. One equation writes the fate of everything that rises and then saturates.

Applications

Epidemiology's SIR model is a set of coupled ODEs on which lockdown decisions were run; chemical kinetics, circuits, and control systems are all ODEs. In AI, Neural ODEs treat a network's "layers" as continuous time — a ResNet's layer-by-layer forward pass is essentially a numerical solve of an ODE, and depth becomes "integration time."

Essence + a question
A differential equation gives the local rule "how you change right now," yet its solution is a global destiny "the whole trajectory" — rule plus initial value, and the future is locked in.
Question: the logistic equation is fully deterministic (no random term), so why can it still describe growth that seems "unpredictable"? Are "deterministic rule" and "predictable future" the same thing? (Day 16, Dynamical Systems, will answer no.)

The Heat Equation

Everything tends toward smoothness, and time cannot run backward
PDE
Intuition

Heat a metal rod red-hot in the middle, and heat flows outward to the cold ends. How fast? It depends on "how much hotter this point is than its neighbors." Mathematics sharpens "the gap from your neighbors" into the second spatial derivative (curvature): a point shaped like a bulging peak (hotter than its neighbors) cools down; one shaped like a dip (colder than its neighbors) warms up. So every sharp corner is rounded, every ripple flattened, until all is uniform.

This is diffusion — not just heat: ink spreading in water, a scent filling a room, all obey the same equation. And it hides a secret: time has a direction. You can watch ink spread out, but you have never seen spread ink gather itself back to a point. Run forward, the heat equation gently smooths everything; run backward, it wildly amplifies the tiniest noise — it gives the "arrow of time" a purely mathematical explanation.

Formal definition
$$\frac{\partial u}{\partial t}=\alpha\,\frac{\partial^2 u}{\partial x^2}$$

$u(x,t)$ is the temperature at position $x$, time $t$; $\alpha>0$ is the thermal diffusivity. The left side is "how temperature changes in time," the right side is exactly that curvature: $\frac{\partial^2 u}{\partial x^2}>0$ (a dip) warms up, $<0$ (a peak) cools down. The equation says one thing — the less flat a spot is, the more urgently it gets smoothed.

t=0 t grows → smoothed x
Why it's beautiful

The heat equation has a stunning explicit solution: an initial point source of heat spreads over time into a normal distribution (a Gaussian bell), growing ever wider with standard deviation proportional to $\sqrt{t}$. This "heat kernel" welds together two seemingly unrelated worlds: deterministic heat diffusion and random Brownian motion. The random jostling of countless molecules obeys, macroscopically, one smooth partial differential equation. Probability and analysis shake hands here.

Applications

Image processing's Gaussian blur is just letting pixel brightness diffuse a few steps under the heat equation; finance's Black-Scholes option-pricing equation is, after a change of variables, a heat equation. Most striking is diffusion generative models (the engine under Stable Diffusion, Sora): during training an image is diffused with noise until "smoothed" into pure noise, and generation then learns to reverse this irreversible process — AI painting is, mathematically, playing a heat-equation movie in reverse.

Essence + a question
The heat equation writes "rate of change is proportional to the gap from your neighbors" in a single line, smooths away every ripple, and hands time an irreversible direction.
Question: if the reverse heat equation amplifies noise and is mathematically "ill-posed," how can diffusion models still stably reverse it and produce clear images? What extra information fills in that irreversibility?

The Wave Equation

One character apart, a reversal of fate
PDE
Intuition

The wave equation looks almost identical to the heat equation — you only swap the left-hand first time derivative for a second. That single change turns the behavior upside down.

A first time derivative (heat equation) says "curvature sets the velocity," so the system slides monotonically toward equilibrium and never comes back. A second time derivative (wave equation) says "curvature sets the acceleration" — like a spring: the farther it is pulled from equilibrium, the stronger the restoring pull, so it overshoots equilibrium, swings to the other side, and swings back, oscillating without rest. A plucked string, a spreading ripple, traveling sound and light are all this "overshoot–rebound" relayed station by station through space. The heat equation makes a disturbance vanish; the wave equation makes it travel.

Formal definition
$$\frac{\partial^2 u}{\partial t^2}=c^2\,\frac{\partial^2 u}{\partial x^2}$$

$u(x,t)$ is displacement (a string's height, air pressure, field strength); $c$ is the wave speed. The left side is acceleration, the right is again curvature. d'Alembert gave a stunningly clean general solution: $u=f(x-ct)+g(x+ct)$ — any right-moving wave plus any left-moving wave, each holding its shape and moving at constant speed. The whole solution space is the free superposition of "a shape going left" and "a shape going right."

Why it's beautiful

Raise the first time derivative to a second, and the system's entire character flips: dissipative becomes conservative, irreversible becomes reversible. The wave equation's energy does not decay in time, and it is time-reversible — play a recording of a wave backward and it is still a valid solution (playing heat diffusion backward is absurd). More astonishing still, Maxwell merged the equations of electricity and magnetism and obtained exactly a wave equation, with speed $c$ equal to the speed of light — light was thereby "predicted" to be an electromagnetic wave. One equation structure unifies sound, water, light, earthquakes, and gravitational waves.

Applications

Acoustics and instrument design, seismic imaging (a CT scan of the Earth), ultrasound and radar, fiber-optic communication all solve the wave equation. The gravitational waves LIGO detected in 2015 are solutions of the wave equation that Einstein's field equations reduce to in the weak field — a verification a century late. Quantum mechanics' Schrödinger equation is a close relative living, via complex numbers, between the wave and heat equations.

Essence + a question
Lift the time derivative from first order to second, and smoothing diffusion becomes undamped oscillation — the same mathematical skeleton, split by one number into worlds of "vanish" and "travel."
Question: why does precisely the "second time derivative" bring time-reversibility and energy conservation? Most fundamental physical equations are second order in time ($F=ma$ too) — what deep symmetry of the universe does this hint at?

Numerical Methods

When no closed-form solution exists, just follow the slope
Numerical Analysis
Intuition

A brutal fact: most differential equations have no solution expressible in elementary functions. The three-body problem, weather, turbulence — you will never get a clean $u(t)=\dots$ formula. But the equation gives you one precious thing: the slope at every point. That's enough.

The crudest Euler method is like walking a mountain trail blindfolded: stand at the current point, feel the slope underfoot (the derivative from the equation), step a small distance $h$ in that direction, land at a new point, feel the slope again, step again — approximating a continuous curve by a polyline. Smaller steps hug the true solution better, but each step's tiny error accumulates, and the polyline drifts off the true curve. The improvement is Runge-Kutta (RK4): before taking a step, probe the slope at several points within it and take a weighted average, letting the step "anticipate" the curve's bend — accuracy jumps by orders of magnitude.

Formal definition
$$y_{n+1}=y_n+h\,f(t_n,y_n)$$

This is Euler's method: $y_n$ is the current value, $f(t_n,y_n)$ the current slope the equation supplies, $h$ the step size. "Current value + slope × step" = the next point; it is the atom of every numerical integrator. RK4 merely replaces the step's slope $f$ with a weighted average of four probes, pressing the error down from proportional to $h^2$ to proportional to $h^5$.

true solution Euler polyline (error accumulates) h
Why it's beautiful

Numerical methods are the reverse-engineering of calculus: calculus pushes a discrete sum to a continuous limit, and numerical methods cut the continuous back into the discrete, making a silent equation speak concrete numbers. It forces you to face a triangle of trade-offs — accuracy, step size, stability: too big a step distorts, too small accumulates error and runs slowly; and with some methods, once the step exceeds a threshold, the error explodes exponentially and diverges outright. This dialogue between continuous and discrete is the backdrop of all scientific computing.

Applications

Weather forecasting, fluid and thermal simulation, game physics engines, rocket trajectories (Apollo's and SpaceX's guidance) all rely on numerically solving differential equations. In AI, every sampling step of a diffusion model is numerically solving a reverse ODE/SDE — the "accelerated samplers" DDIM and DPM-Solver are, in essence, smarter Runge-Kutta.

Essence + a question
Can't solve it in closed form? Then walk step by step along the slope the equation gives — numerical methods cut the continuous back into the discrete, at the eternal price of trading off accuracy, step size, and stability.
Question: if smaller steps are always more accurate, why not take an infinitesimal step? (Hint: floating-point round-off error and computational cost strike back from the other end — see Day 62.)

Deeper Thoughts

Open problems · toward the boundary and across disciplines
Why is the heat equation irreversible but the wave equation reversible — only because of the order of one time derivative?
The key is time reversal $t\to -t$. The second derivative $\partial^2/\partial t^2$ is unchanged under this substitution (negative times negative is positive), so the wave equation stays intact — a wave played backward is still a valid solution, energy conserved. The first derivative $\partial/\partial t$ flips sign, so the forward-smoothing heat equation, run backward, becomes a noise-amplifying "anti-diffusion" that is ill-posed in math and never happens in physics. An equation's time symmetry directly decides whether the world it describes has an "arrow of time" — a difference of one sign splits reversible and irreversible universes.
How do diffusion models "reverse" a heat equation that should be irreversible?
The trick is that they don't actually invert that ill-posed reverse equation; they learn something extra: a neural network trained on vast images learns to estimate, at each noise level, "which direction of denoising most resembles a real image" (the gradient of the data distribution's log-density, the score). This prior stolen from data fills exactly the information the irreversible process lost, turning the ill-posed inversion into a reverse stochastic differential equation with a unique, stable solution. Irreversibility is not violated — the knowledge in the data pays the bill for time's reversal.
What is a "well-posed" problem, and why is it the lifeline of applied mathematics?
Hadamard proposed three conditions: a solution exists, is unique, and depends continuously on the initial data (a small perturbation of input causes only a small perturbation of output). The third is decisive: the reverse heat equation fails right here — a tiny measurement noise in the initial data is amplified into a towering wave in the solution, so any numerical computation is untrustworthy. Real-world "inverse problems" (recovering a sharp image from a blurry photo, inferring underground structure from surface data) are mostly ill-posed, and must be tamed by regularization — injecting prior assumptions by hand. Well-posed or not decides whether a model can actually be computed.
Why is numerical error doomed to run wild in chaotic systems, and does that make long-term prediction impossible?
Chaotic systems are exponentially sensitive to initial conditions (the butterfly effect, see Day 16). The tiny round-off error a numerical method introduces each step is equivalent to continually injecting new initial-condition perturbations, which get amplified exponentially. So no matter how precise the algorithm or how small the step, past a certain "predictability horizon" the numerical trajectory and true solution must diverge beyond recognition — this is why weather forecasts fail at about two weeks. But this is not total ignorance: a single trajectory can't be computed, yet its statistical regularities (the shape of the attractor, long-term averages) are often stably computable. A deterministic equation → an individual that can't be predicted long-term + a whole that can — is the deepest lesson differential equations leave us.