Visual Thinking in Mathematics

Peel the outer 10% off the radius of a three-dimensional orange and $0.9^3\approx 73\%$ of the flesh survives. Do the same in 100 dimensions: $0.9^{100}\approx 0.003$. A high-dimensional orange is almost entirely peel — not because the orange is strange, but because volume is governed by the $n$-th power of the radius.

Something stranger still: pick two directions at random in high dimensions and they are almost certainly close to perpendicular. In two dimensions the angle is uniform; in a hundred, it clusters tightly around 90°. So the right posture is not to strain at picturing a hundred-dimensional cube — nobody can — but to replace the instinct.

The volume of the $n$-dimensional unit ball is $V_n=\pi^{n/2}\big/\Gamma(\tfrac{n}{2}+1)$, where $\Gamma$ is the continuous extension of the factorial. The denominator outruns the numerator, so $V_n$ peaks near $n=5$ and then falls to zero: the high-dimensional unit ball occupies almost none of the cube that contains it. The volume has fled to the corners.

The sharper statement is concentration of measure: on the sphere, any 1-Lipschitz function $f$ (move the input a little and the output moves at most as much) satisfies

The point is that $n$ sits in the exponent: the higher the dimension, the closer any "gentle" function is to a constant.

We assume dimension is quantity stacked — one more axis, merely more room. What the above shows is that dimension is a parameter with a phase transition: low-dimensional intuition does not blur gradually as $n\to\infty$, it is replaced wholesale by a different regime. The lovely part is that the new regime is not chaos but an unusual tidiness. The law of large numbers is exactly concentration of measure on a product space: the most basic theorem in probability is, underneath, a fact of high-dimensional geometry.

• Embeddings: the cosine similarity of two random 768-dimensional vectors lands, with overwhelming probability, in a narrow band around 0 — so 0.3 is already a strong signal. Setting thresholds by low-dimensional instinct misjudges badly.
• Diffusion models: samples from a $d$-dimensional standard Gaussian sit almost entirely on a thin shell of radius $\sqrt{d}$, not near the origin. Noise schedules and sampling paths are built on that shell.
• Nearest-neighbour failure: the precise content of the curse of dimensionality is that the ratio of farthest to nearest distance tends to 1, which degrades KNN and ball trees together.

Reducing dimension is projecting, and projecting is casting a shadow. A hand becomes a rabbit on the wall — enormous information is lost, but the part you cared about survives. The real question is: which wall.

PCA picks the wall on which the shadow spreads widest, the direction of maximum variance. What comes next is the surprise: if all you care about is who is near whom, then any wall will do — a random projection is nearly as good as the optimal one.

The Johnson–Lindenstrauss lemma: given $N$ points in $\mathbb{R}^d$ and a tolerance $\varepsilon$, there is a linear map $f:\mathbb{R}^d\to\mathbb{R}^k$ such that as soon as

every pair satisfies $(1-\varepsilon)\|x-y\|\le\|f(x)-f(y)\|\le(1+\varepsilon)\|x-y\|$.

The crux: there is no $d$ in $k$. The target dimension depends only on the logarithm of the number of points and on the accuracy you want — nothing about how high the original space was. A million points, whether they live in $10^3$ or $10^9$ dimensions, need only a few thousand dimensions at $\varepsilon=0.1$. And the way to construct $f$ turns out to be: take a random Gaussian matrix.

Three surprises at once. First, the dimension needed to preserve distances does not depend on the original dimension — "high-dimensional" is charged nothing here. Second, the random construction is essentially optimal: good directions are abundant and bad ones exponentially suppressed, so there is no need to hunt. Third and finest, it rewrites the bad news of Concept 1 into a tool — precisely because projected lengths hug their expectation, one can take a union bound over all $N^2$ pairs at once. The same geometric fact is both the curse and the blessing of dimensionality.

• Vector search: random projection and LSH sit underneath libraries like FAISS — compress first, index second, with JL guaranteeing recall does not collapse.
• Different objectives: PCA is linear and preserves global variance; t-SNE and UMAP preserve local neighbourhoods, which is why distances between clusters and the size of clusters in a t-SNE plot are essentially unreadable — the most widely misread class of figure in current practice.
• Interpretability: visualising a hidden layer is always a projection. How much of the structure you see comes from the data and how much the method manufactured must be kept apart.

Mathematical animation is often taken for pedagogical decoration, but its core is a genuine mathematical object: continuous deformation. When 3Blue1Brown teaches linear algebra he does not draw a matrix as a table of numbers; he stretches and rotates the whole grid — because a matrix was never a noun, it is a verb. A static picture can only show the verb's result; animation shows the verb.

Then the stronger step: the deformation can itself be the proof. If A can be carried continuously into B while some quantity never changes, that quantity must agree on A and B. That is the entire idea of a topological invariant.

Two continuous maps $f,g:X\to Y$ are homotopic if there is a continuous

$X$ is the thing being deformed, $Y$ the space it lives in, $[0,1]$ is the time axis and $H(\cdot,t)$ is frame $t$ — so $H$ is the animation. A homotopy invariant is any quantity that does not budge under all such animations.

Take the inscribed rectangle problem: does every closed plane curve carry four points forming a rectangle? Assemble all unordered pairs of points on the curve into a space — since $(a,b)$ and $(b,a)$ count as one point, that gluing produces exactly one twist, and the space is a Möbius band. Now map each pair into $\mathbb{R}^3$: record the midpoint's position together with the distance between the two points. A rectangle is precisely two distinct pairs sharing a midpoint and a distance — a self-intersection of this map. But the boundary of the Möbius band is the original curve and must lie flat in the plane, and topology forbids the interior from embedding without self-intersection. So the rectangle must exist.

Not one equation appears in that argument; it is all shape. This is visual thinking as an engine of discovery in its purest form: it does not illustrate a proof, it is one.

• Manim, 3Blue1Brown's animation library, replaces keyframes with parametrised families of continuous transformations — what you write is not an animation but $H$.
• Representation learning: what a classifier does is pull two entangled manifolds continuously apart until they are linearly separable. Chris Olah's classic visualisations show that a network's depth is roughly the number of deformation steps it is allowed.
• Homotopy continuation: to solve a hard nonlinear system, start from an easy system whose solution you know, deform continuously to the target, and track the solution along the way — used in circuit simulation and robot inverse kinematics.

Anscombe's quartet: four datasets agreeing to two decimal places in mean, variance, correlation and regression line, yet plotting into four unrelated pictures. Summary statistics are lossy compression; the plot is the part that was discarded. The later Datasaurus pushes the joke to its limit — the same statistics can draw a dinosaur.

But plots deceive too, and the deception can be quantified. The eye reads position most accurately, length next, then angle, area, volume and colour saturation. So a pie chart (angle plus area) is systematically worse than a bar chart, and a 3-D bar chart (volume) is all but guaranteed to mislead.

The formal frame for visualisation is the grammar of graphics: a chart is a map from data variables to visual channels,

and each channel carries its own perceptual accuracy — Cleveland and McGill measured the ranking above in controlled experiments. "Designing a chart" then has an exact meaning: choose one map among all of them, and give the most important variable the most accurate channel.

This turns a question of taste into a channel problem. Data must cross a noisy, nonlinearly responding channel — the human visual system — before it reaches understanding, and design is the choice of code for that channel. It is isomorphic to information theory: there is capacity, there is distortion, there are better and worse codes. A misleading chart is not "ugly", it is a coding error: a high-importance variable pushed into a low-accuracy channel. Aesthetics acquires a falsifiable criterion.

• Loss landscapes: the flat-or-sharp appearance of a 2-D slice can be distorted arbitrarily by rescaling parameters, which is exactly what Li et al.'s filter normalisation exists to fix so the pictures are comparable across models.
• Attention heatmaps: colour is among the least accurate channels and judgements of saturation depend heavily on context — the same attention map under a different colormap yields a different "conclusion".
• Model evaluation: an ROC curve maps two rates onto position channels and preserves far more than a single AUC number; one AUC value corresponds to wildly different curve shapes. That is Anscombe, restated for models.