Day 47 · 2026.08.08

Model Theory & Foundations

However much a language can say, that is how many worlds there are
"universal algebra + logic = model theory" — C. C. Chang & H. J. Keisler, Model Theory

First-Order Logic & Completeness

Why proof and truth coincide
Logic · Foundations
Intuition

Write down the group axioms — associativity, identity, inverses. Those few lines of symbols point at nothing on their own; they are a sieve: integer addition passes, invertible matrices pass, addition on the naturals does not. Every structure that passes is a model of the axioms.

So mathematicians work on two levels. The syntactic level: rearrange symbols by rules, step by step — finite, mechanical, delegable to a machine ($T\vdash\varphi$). The semantic level: go into every model and check whether the sentence holds — a single $\forall x$ demands checking infinitely many $x$, which no human finishes ($T\models\varphi$). In 1929 Gödel proved: the two levels agree exactly.

Do not confuse this with incompleteness: completeness (1929) says the rules of inference suffice; incompleteness (1931) says the axioms of arithmetic do not. The first is about logic, the second about one particular theory.

syntax · finite semantics · infinite axioms T ↓ finite derivation T ⊢ φ machine-enumerable every model M ⊨ T T ⊨ φ cannot be checked one by one completeness theorem
Finite and enumerable on the left, infinite and untraversable on the right — completeness declares the two have the same extension.
Formal statement
$$T\models\varphi\quad\Longleftrightarrow\quad T\vdash\varphi$$

The left side is semantic entailment: every model of $T$ makes $\varphi$ true. The right side is syntactic derivability: some finite formal derivation exists. The $\Leftarrow$ direction is soundness, and it is easy; all the content sits in $\Rightarrow$ — there is no first-order sentence that holds in every model yet can never be proved.

Why it is beautiful

Truth is a statement about infinity, a proof is a finite object, and the two turn out to have the same extension. This is the first precise version of "the infinite captured by the finite," and it is the bedrock of automated theorem proving — a machine only shuffles symbols, and completeness guarantees that shuffling symbols misses no universal truth.

The flip side surfaces immediately: since a proof is finite, it can invoke only finitely many axioms. The world-building machine of the next card grows straight out of that remark.

Applications

Relational query languages are first-order logic made industrial: relational algebra corresponds one-to-one with first-order formulas, and SQL's WHERE and EXISTS are its connectives and quantifiers; Datalog and Prolog share the ancestry. In formal verification, Lean and Coq rest on stronger type theory, but the layer that actually discharges subgoals automatically — SMT — still runs on first-order logic.

One line · a question
A formal proof is not a substitute for truth; completeness is what makes it truth's equivalent.
Question: if first-order logic is complete, why is automated theorem proving still hard? What lies between "a proof exists" and "a proof is found"?

The Compactness Theorem

A machine that builds worlds out of nothing
Model Theory
Intuition

If a pile of sentences contradicts itself, the contradiction is already exposed within finitely many of them — the derivation of the contradiction is itself finite and only has time to use finitely many premises. Turned around, that is compactness: if every finite subset has a model, the whole (possibly infinite) set has a model.

The principle sounds like a triviality and is in fact a machine for building worlds. Want an "infinitely large natural number"? Add to the language of arithmetic a constant $c$ and infinitely many axioms $c>0,\ c>1,\ c>2,\dots$. Take any finite batch: the harshest one merely says $c>n$, so letting $c$ be $n+1$ satisfies it — the ordinary naturals are a model. Compactness therefore asserts that the whole set has a model, and inside it lives an element greater than every standard natural number. The conclusion is unsettling — first-order language cannot say "finite," and cannot pin down "this is the natural numbers."

infinite axiom set T c > 0 c > 1 c > 2 c > n any finite batch → take c = n+1, ℕ is a model compactness a model of all of T c: infinite all standard naturals sit to the left
Every finite fragment is satisfiable in the ordinary naturals, so the whole set must have a model — and that model must contain a non-standard element.
Formal statement
$$\text{every finite } T_0\subseteq T \text{ has a model}\ \Longrightarrow\ T \text{ has a model}$$

"Compact" is not a metaphor: take complete theories as points and the sets carved out by single sentences as basic open sets, and the resulting topological space (the Stone space) is literally compact — the theorem is the logical translation of "finite subcover."

Why it is beautiful

It converts the unremarkable observation that proofs are finite into existence from nothing. Löwenheim–Skolem is the same machine's other exit: a theory in a countable language with any infinite model has models of every infinite cardinality. So if ZFC is consistent it has a countable model, even though ZFC internally proves that uncountable sets exist. Skolem's paradox is no contradiction: "uncountable" is a verdict passed inside the model, meaning the bijection is absent from the model; seen from outside, it is of course available. How many there are depends on where you stand to count.

Applications

The de Bruijn–Erdős theorem — an infinite graph whose every finite subgraph is $k$-colorable is itself $k$-colorable — falls straight out of compactness. The converse view is more interesting: compactness fails on finite models, which is exactly why database theory had to build a separate "finite model theory." Real tables are finite, and there the workhorse theorems of classical model theory are all void.

One line · a question
Anything a first-order language can say is subject to compactness — and so it can never pin down an infinite structure.
Question: safety properties of distributed systems are often argued as "holds for all finite executions." Under what conditions does compactness let you lift that to infinite executions?

Non-standard Analysis

The legal return of the infinitesimal
Analysis · Logic
Intuition

The infinitesimal $dx$ of Newton and Leibniz — smaller than every positive number yet not zero — was mocked by Bishop Berkeley as "the ghost of a departed quantity" and expelled in the nineteenth century by $\varepsilon$–$\delta$. In 1960 Robinson used the machine of the previous card to bring it back: add to the theory of the reals a constant $\epsilon$ and infinitely many axioms $0<\epsilon<1/n$. Every finite subset is plainly satisfiable, so an ordered field containing genuine infinitesimals exists — the hyperreals $^*\mathbb{R}$.

In $^*\mathbb{R}$ the derivative is no longer a limit; it is the difference quotient itself: compute $\frac{f(x+\epsilon)-f(x)}{\epsilon}$ and take its standard part (the nearest real number). Continuity no longer needs the three-quantifier dance either: if $x$ and $y$ are infinitely close then so are $f(x)$ and $f(y)$.

0 1 2 ⋯ the reals ℝ ⋯ ω = 1/ε infinite 0 ε −ε the halo of 0
Every real is wrapped in a halo of infinitesimals; taking the standard part collapses the whole halo back onto its center.
Formal statement
$$\mathbb{R}\models\varphi\quad\Longleftrightarrow\quad{}^*\mathbb{R}\models\varphi\qquad(\varphi\ \text{a first-order sentence})$$

This is the transfer principle: any property expressible in first-order language holds in the reals exactly when it holds in the hyperreals. Every reason $^*\mathbb{R}$ is usable lives here — it is automatically an ordered field, and every real function automatically extends. "First-order" is the crucial restriction: "every subset with an upper bound has a supremum" quantifies over subsets, is second-order, does not transfer, and indeed $^*\mathbb{R}$ is not complete. When old-style infinitesimal arguments went off the rails, this is the rail they left.

Why it is beautiful

An intuition exiled for a century and a half was rehabilitated not by analysis but by logic. Better still, it explains the history: why those "non-rigorous" arguments so often produced correct results — they were informal instances of the transfer principle, and transfer is a theorem. It also marks precisely where such arguments break.

Applications
  • Stochastic analysis: Loeb measure represents Brownian motion as a "hyperfinite" random walk, so combinatorial counting arguments can be applied to a continuous model.
  • Shorter proofs: Tao has repeatedly shown how to trade an asymptotic statement about "$n\to\infty$" for the algebraic statement "fix an infinite $n$," at which point the layered $\varepsilon$ bookkeeping evaporates.
  • A computational cousin: the dual numbers $a+b\epsilon$ of automatic differentiation ($\epsilon^2=0$) are not hyperreals — no order, no transfer principle — but they too let an infinitesimal genuinely participate in arithmetic. Forward-mode differentiation in JAX and PyTorch is exactly this.
One line · a question
Infinitesimals were never non-rigorous; there was simply no language yet able to carry them.
Question: analysis over $^*\mathbb{R}$ proves exactly the same theorems about the reals as classical analysis does — it is a conservative extension. If it buys not a single new theorem, where does its value actually lie?

Decidability: the Computability of a Theory

Quantifier elimination and the price of expressiveness
Computability · Logic
Intuition

Incompleteness says "PA contains sentences that are true but unprovable." The question here is a more engineering one: given a theory, is there an algorithm that decides whether an arbitrary sentence is one of its theorems?

The watershed is astonishingly narrow. Arithmetic on the naturals with addition only (Presburger arithmetic) is decidable; add multiplication and you have PA, which is instantly undecidable; yet the first-order theory of the reals with both addition and multiplication (real closed fields) was proved decidable by Tarski in 1951 — the seemingly larger structure is the tamer one.

What matters is not how many objects there are but whether the integers can be defined inside the theory: with the integers you can encode a Turing machine, and with that you have smuggled in the halting problem. No first-order formula in a real closed field picks out the integers, so it stays decidable. Undecidability has exactly one source — the capacity for self-reference.

can it encode a Turing machine? decidable undecidable propositional Presburger RCF PA FO validity ZFC expressiveness → NP-complete doubly exponential halting reduces into it decidable ≠ feasible
The watershed is not the size of the objects but whether computation can be simulated inside the structure.
Formal statement
$$\exists x\,(ax^2+bx+c=0)\iff\big(a\neq0\wedge b^2-4ac\ge0\big)\vee\big(a=0\wedge(b\neq0\vee c=0)\big)$$

Tarski's method is quantifier elimination: every formula can be rewritten as an equivalent quantifier-free one. The line above is the smallest example — the left side searches over infinitely many $x$, the right side needs only finitely many arithmetic tests on the coefficients. Quantifier-free truth can be computed directly, so eliminating quantifiers one by one renders the whole theory decidable.

Why it is beautiful

Decidable does not mean easy: the decision problem for Presburger arithmetic has a doubly exponential lower bound, and a whole wilderness of complexity separates decidable from feasible. The real beauty is the watershed itself — it lands precisely on "can this structure simulate computation inside itself?" Gödel (incompleteness), Turing (halting), and Tarski (undefinability of truth) converge here into one sentence: whatever can refer to itself loses decidability.

Applications

SMT solvers (Z3, CVC5) are exactly these decidable fragments made industrial: linear integer arithmetic, bit-vectors, arrays and uninterpreted functions each get a decision procedure, then the procedures are combined. Compiler verification (CompCert), kernel verification (seL4), symbolic execution and program synthesis all run on top of them. The line also runs backwards into design — Datalog endures precisely because it deliberately stops on the decidable side: choosing a specification language for a system is really choosing a point on the trade between expressiveness and decidability.

One line · a question
Expressiveness is conserved: every extra thing a language can say costs you something you can no longer decide.
Question: a type system is a theory of programs. Why must dependent types (expressive enough to approach first-order arithmetic) surrender decidable checking or inference, while Hindley–Milner need not?

Going deeper

If first-order language cannot pin down the naturals, why do we still believe "the natural numbers" is a determinate object?
The second-order induction axiom quantifies over all subsets and does pin down $\mathbb{N}$ uniquely (categoricity), at the cost that second-order logic has no complete proof system — you fix the object but lose the ability to reason about it mechanically. Categoricity and completeness cannot both be had; that is logic's own uncertainty principle. A Platonist says the standard model objectively exists and language merely cannot reach it; a formalist says "standard" is only a relative notion selected from a metatheory. What is certain is just this: whatever determinacy there is, it does not live in first-order syntax.
Is the connection between model theory and machine learning real, or only an analogy?
It is real. Shelah's stability theory classifies theories by whether a formula can encode arbitrarily long binary orders inside a structure, and the combinatorial parameters that fall out match the dimensions of learning theory one for one: NIP theories correspond to finite VC dimension (PAC learnability), stable theories to finite Littlestone dimension (online learnability). Work by Chase, Freitag and others aligned two taxonomies that had developed independently for half a century. The shared intuition: complexity that admits a finite description is equivalent to the inability to encode arbitrarily complex patterns.
Why does finite model theory have to start over from scratch?
Because both pillars of classical model theory assume infinite models are allowed. Compactness fails outright — each finite subset of the family "the model has at least $n$ elements" has a finite model, while the whole family does not. Completeness fails too: the set of first-order sentences valid on finite structures is not recursively enumerable (Trakhtenbrot's theorem). What is lost buys something else, though: over finite structures, logical expressiveness corresponds exactly to computational complexity, and Fagin's theorem shows NP is precisely the class of properties expressible in existential second-order logic. This is descriptive complexity — there, P vs NP becomes a question purely about what a language can say.