Day 30 · 2026.07.22

The Mathematics of AI

Peel open a neural network and inside are four classical mathematical ideas
"Deep learning is not alchemy — it is calculus, linear algebra, and geometry, engineered." — adapted from Yann LeCun

Backpropagation

Making the chain rule scale
Automatic Differentiation
Intuition

A deep network is hundreds of nested functions: $f_n(\cdots f_2(f_1(x)))$. Training must answer one question — "if I nudge one of these billions of knobs a hair, by how much does the final loss change?" — i.e. the partial derivative of the loss with respect to each weight.

The naive way is to test each knob separately: billions of forward passes, astronomical. Backprop's insight: run forward once, cache each layer's intermediate result, then propagate the "blame" backward layer by layer, starting from the loss. Each layer needs only two things — its own local derivative, and the gradient handed down from above — and their product is that layer's contribution, while the gradient passes on to the next layer. Like assigning blame down a chain of command: ask each link "how much did you amplify the error?", one pass, everyone accounted for, nothing recomputed.

$$\frac{\partial L}{\partial w_\ell}=\frac{\partial L}{\partial a_n}\cdot\frac{\partial a_n}{\partial a_{n-1}}\cdots\frac{\partial a_{\ell+1}}{\partial w_\ell}$$
Formal definition

This is the chain rule: $L$ is the loss, $a_\ell$ the activation of layer $\ell$, $w_\ell$ its weights. The product is really a string of Jacobian matrices multiplied together. Everything hinges on the order: right to left (reverse mode) — compute the "loss-over-output" vector at the left end first, then left-multiply step by step — every step is just "vector × matrix," never building a big matrix explicitly. So the gradient of all parameters costs about one forward pass.

Why it's beautiful

The same product of terms, run in the opposite direction, collapses the cost from "number of parameters × forward" to "one forward." This is the heart of reverse-mode automatic differentiation — mathematically, the "adjoint." It's no black magic but an older duality: to get one output's sensitivity to many inputs, walk backward. Exactly this $O(n)$-instead-of-$O(n^2)$ gap turns training billions of parameters from "impossible" into "an overnight job."

Applications

Every trained neural network relies on it — PyTorch's and JAX's autograd automatically differentiates any computation graph in reverse. The same idea is just as sharp outside nets: the adjoint method in weather and fluid inversion, Pontryagin's principle in optimal control, differentiable rendering pushing gradients through a graphics pipeline, even reasoning backward from a gradient to a molecular structure. Wherever you have "many inputs, one scalar objective," backprop is the optimal solver.

Essence + Question
Anyone can write the chain rule; backprop's genius is multiplying it backward — squeezing the cost of billions of gradients back down to one forward pass.
In a very deep net the gradient multiplies hundreds of numbers along the way: all slightly below 1 and it decays exponentially (vanishing gradients); all slightly above 1 and it explodes. Why do ResNet's "skip connections" ease this? Hint: they slip an identity term $+1$ into the product.

Attention

A differentiable soft dictionary
Linear Algebra
Intuition

Read "the kitten chased its tail because it was bored." To know what "it" refers to, you scan back over the text, find the most relevant word, and pull its meaning in. Attention makes this mathematical: each word sends out a query (Q), every other word holds up a key (K) answering "am I relevant to you?", and the more relevant ones hand over more of their value (V).

So attention is a soft dictionary: an ordinary dictionary retrieves by an exact key, attention gives every key a continuous weight by similarity and takes a weighted average. And it is differentiable — how well it retrieves can be optimized by the gradient — so "where to look" goes from hand-designed to something the network learns itself.

$$\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\!\left(\frac{QK^{\top}}{\sqrt{d}}\right)V$$
Formal definition

$Q,K,V$ are three linear projections of the same batch of word vectors (query, key, value). $QK^{\top}$ uses the dot product to measure each query's similarity to each key, giving a table of scores; dividing by $\sqrt{d}$ keeps large dimensions from blowing up the dot products and saturating the softmax; softmax squashes each row of scores into weights summing to 1; multiplying by $V$ is "mix all the words' contents, weighted by relevance."

Why it's beautiful

It reduces the fuzzy cognitive act of "retrieving memory by content" to two matrix multiplications and a softmax — clean, differentiable, and naturally suited to GPU parallelism. Where an RNN must go word by word and lets information decay down a long chain, attention lets any two words reach each other in one step, a constant distance no matter how far apart. This operator, writable in a few lines, holds up the entire era of Transformers and large language models.

Applications

GPT and BERT model language with it, the Vision Transformer (ViT) treats image patches as "words," AlphaFold uses it to capture long-range pairings between amino acids and infer protein folding, and diffusion models use cross-attention to let text steer image generation. "Which elements should influence each other" is answered, across nearly every modality, by the same formula.

Essence + Question
Attention = a differentiable soft dictionary: use similarity as weights for a weighted average, letting the network learn where to look at each moment.
$QK^{\top}$ is an $n\times n$ matrix — double the sequence and compute and memory quadruple. This is exactly the long-context bottleneck. To bring that quadratic cost down toward linear, what would you sacrifice? (Think: does every word really need to see every other word exactly?)

Networks = Universal Approximators

Universal Approximation · building any shape from small steps
Approximation Theory
Intuition

A single neuron does something simple: weight and sum the inputs, then pass through a nonlinearity — the output approximates a soft step or a little bump. One bump is dumb, but stack many bumps of varying height, width, and position and you can carve out any undulating curve — like Lego bricks approximating a mountain ridge, or Fourier stacking sine waves into a square wave.

The universal approximation theorem says exactly this: with enough hidden neurons, a network can approximate any continuous function to any precision you want. A neural network learning "anything" is no mysticism — it is essentially a tunable function-Lego machine, and learning is adjusting each brick's height and position.

target f many soft steps stacked
Formal definition

Cybenko (1989) and Hornik's theorem: finite sums of the form $\sum_{i} c_i\,\sigma(w_i\!\cdot\! x+b_i)$ ($\sigma$ a nonlinearity like a sigmoid) are dense in the space of continuous functions on a compact set — they can approximate any continuous $f$ arbitrarily well. But the crux: the theorem only guarantees that such parameters exist; it says nothing about how many neurons (possibly exponentially many), and nothing about whether gradient descent can find them.

Why it's beautiful

It reconnects the fashionable neural network to a three-century classical lineage of approximation — Weierstrass, Fourier — the network is just another set of basis functions. Better still, it separates three things often conflated: can it represent (the theorem — yes), can it learn (optimization), can it generalize (statistics). Deep learning's real mystery is not the first but this: why does an over-parameterized network with far more parameters than data, trained by plain SGD, nonetheless find solutions that generalize well?

Applications

It is the license to use a network as a "universal function fitter": if a problem can be written as an "input→output" function, there's reason to try. The same logic explains why deep often beats wide — some functions need exponentially many neurons in a shallow net but only polynomially many when made deeper; depth buys efficiency through composition.

Essence + Question
"A network can fit any function" is a floor the theorem guarantees, not a miracle; the real miracle is that SGD actually finds one — and that it generalizes.
Universal approximation already holds for a "wide enough" single-layer shallow net, so why do we still want "depth"? If deep and shallow are equal in expressive power, is depth's real payoff expressiveness — or learnability and generalization?

The Manifold Hypothesis

High-dimensional data really lives on a thin sheet
Geometry & Topology
Intuition

A $100\times100$ grayscale image lives in a $10000$-dimensional pixel space. But scatter a handful of random pixels and you will never land on a human face — real images occupy only a thin, curved sheet within that vast space, of nearly zero measure. The manifold hypothesis says: real high-dimensional data (images, speech, text) does not fill the whole space but concentrates on a much lower-dimensional curved surface (a manifold).

An analogy: a ribbon crumpled into a room is intrinsically one-dimensional (walking along it there's only "forward/back"), yet embedded in three dimensions. Learning is flattening that crumpled sheet and finding its intrinsic coordinates — grasp the sheet's shape, and a ten-thousand-dimensional problem collapses into a few-dozen-dimensional one.

Formal definition

Assume high-dimensional data lies approximately near a low-dimensional manifold $\mathcal{M}\subset\mathbb{R}^D$ whose intrinsic dimension $\dim\mathcal{M}=d\ll D$. Representation learning is finding a set of coordinates (a chart / embedding) for that manifold, compressing the $D$-dimensional observation into a $d$-dimensional latent vector with almost no loss.

Why it's beautiful

It resolves the paradox of the curse of dimensionality: in theory learning in $10000$ dimensions needs an astronomical number of samples, yet in practice a few million images suffice — because the effective dimension is low. It ushers differential geometry and topology into deep learning: smoothly interpolating between two points in latent space corresponds to "walking along the sheet" of the manifold, so a face can shift continuously from a smile to a frown without passing through a field of snow. The shape of the data is itself a form of prior knowledge.

Applications

Autoencoders compress data to low dimensions and reconstruct it; diffusion models can be read as learning to "push noise that has strayed off the manifold back onto it," step by step generating realistic images from pure noise; t-SNE and UMAP spread a high-dimensional manifold onto two dimensions for the eye; latent-space interpolation and image editing all rest on the geometric intuition of "moving on the manifold."

Essence + Question
High-dimensional data doesn't fill high-dimensional space — it lives on a low-dimensional curved sheet; learning is finding that sheet's shape and coordinates.
If the data manifold is itself "twisted" (non-orientable, like a Möbius strip), or pieced together from fragments of differing dimension, what goes wrong when you describe it with a single global low-dimensional coordinate system? Does this hint that the single-latent-space assumption is sometimes too naive?

Going Deeper

The four ideas — backprop, attention, universal approximation, manifolds — belong to calculus, linear algebra, approximation theory, and geometry. How do they mesh into one machine inside a training loop?
One way to string it together: universal approximation guarantees the network — a "function-Lego machine" — can in principle represent the mapping we want; operators like attention decide how the bricks are assembled and how information flows between elements; backpropagation supplies the efficient gradient engine so SGD can actually tune the bricks into place; and the manifold hypothesis explains why all of this is feasible in high dimensions — the low-dimensional structure of real data makes learning that "should" need astronomical samples possible. Expressiveness, optimization, geometric prior — three pillars, none dispensable.
Universal approximation says "can represent" but is silent on "can learn" and "can generalize." Where is modern deep-learning theory stuck?
Stuck on the latter two. Classical statistics says "more parameters than data must overfit," yet over-parameterized networks generalize well — a crack that spawned double descent, implicit regularization, the neural tangent kernel (NTK). The core conjecture: SGD isn't searching blindly but is steered by some implicit bias (favoring "flat," "simple" solutions), landing exactly in well-generalizing regions. Why? There's still no fully satisfying answer — this is deep learning's most honest frontier.
Attention's quadratic cost $O(n^2)$ is a hard wall for long context. What structural assumption is each attempt to get around it betting on?
All of them bet "relevance is sparse or low-rank." Sparse attention bets each word only needs to look at a few others; linear-attention / kernel methods bet the attention matrix is approximately low-rank, so you can compute $K^{\top}V$ first and avoid the $n\times n$; state-space models (Mamba and kin) bet the history can be squeezed into a fixed-size state. Every speedup is a structural wager: bet that the interactions you drop don't matter — right, and you're efficient; wrong, and you lose capability.
These four pieces of math are all decades old (the chain rule from the 17th century, the approximation theorem 1989, manifolds classical geometry). Why did AI's explosion only arrive in the last decade?
Because "the idea" is only one of three conditions. The mathematics of approximation and differentiation was long in place, but compute (GPUs making large matrix multiplies cheap) and data (the internet providing vast samples on the manifold) were missing. Backprop was known in the 1980s, yet had to wait for data and hardware to arrive together before it showed its power. Breakthroughs are often not a new theorem but an old idea meeting the material conditions that let it scale — the attention formula itself is simple enough to have been written down earlier.