Did Large Models Learn Grammar

Replace every content word with an invented one and the model still picks the right verb form

The troublemaking noun in the middle has a name: an attractor. Bock and Miller had people produce sentences from materials in front of them and found a very stable asymmetry — a singular subject followed by a plural noun draws visibly more errors, while the reverse draws far fewer. That asymmetry has been replicated in several languages across three decades.

In 2016 Linzen, Dupoux and Goldberg handed the same problem to a machine: real sentences pulled from Wikipedia, and a recurrent network (a network that reads one word after another in order) trained to say whether the verb should be singular or plural. With no intervening noun it almost never erred; add attractors one at a time and the error rate climbs step by step — a shape startlingly like the human one.

But their real point was a different one: swap the training objective from "predict the number" to "predict the next word" — the only thing every large model today does — and the same network gets visibly worse. Their conclusion was blunt: the language-modelling signal alone is not enough.

Two years later that was pushed back on. What Gulordava and colleagues trained in 2018 were plain language models, in four languages: English, Italian, Hebrew, Russian. The trick was in the test sentences — every content word replaced by an invented word of the same class, as in "The colorless green ideas I ate with the chair sleep furiously". Meaning is destroyed; only the skeleton and the endings survive. The models still picked the right number reliably, and in Italian came close to human intuitions.

That substitution is the hinge of the whole enterprise. Without it a model can answer correctly while knowing nothing about structure: a pairing like "key … is" is common in the corpus anyway, and the statistics from the issue on a word being known by the company it keeps would carry much of the load. Put invented words in and the co-occurrence crutch is gone. The Italian case looks like this: le chiavi dell'armadio sono sul tavolo ("the keys of the wardrobe are on the table"), where the verb sono (are, plural) agrees with chiavi (keys, plural) and not with the adjacent armadio (wardrobe, singular).

The evidence strength has to be said plainly: four languages, but all four richly inflected, so this is no typological sample. Nor was the tension between the 2016 and 2018 results resolved by one side being right — data volume, network size and test material all differed. The fair summary today is that the bigger the model the more reliably it passes tests like these, so the disagreement has shifted wholesale, from whether it can to what it means that it can.

A probe can read the sentence skeleton out of a model — but give it random labels and it learns those too

Behavioural tests see only what goes in and what comes out. To look inside, the method is called probing: take the model's internal states as raw material, train a very small readout on top of them, and see whether some kind of information can be recovered. In 2019 Hewitt and Manning built an elegant version. Sentence structure can be thought of as a set of distances: any two words are some number of steps apart, close if they attach directly, far if the link runs through several layers. They found a single linear transformation — one stretch and rotation applied uniformly to every vector — under which the distance between two word vectors matches that number of steps. On BERT and ELMo (names of two models common at the time) a good part of the hierarchy could be recovered; on the control representations it could not. The same year Tenney and colleagues found the layers behaving like a pipeline: earlier ones handle word classes, middle ones sentence structure, coreference surfaces later.

Also in 2019, Hewitt and Liang fitted the method with a mirror that exposes fakes, called the control task: assign every word type a random label. Such labels carry no linguistic information, so the only way to learn them is for the probe itself to memorise them. Several probes popular at the time scored high accuracy on the random labels too. So "the probe scores high" says nothing by itself — what counts is the gap: real task minus control task, which they call selectivity.

The layer that matters more is this: decodable is not the same as used by the model itself. In 2019 Lakretz and colleagues opened up a recurrent language model down to single neurons and found that long-distance number information is carried mostly by a very small number of units, themselves governed by another set of units that track structure; switch those off and long-distance agreement collapses. Breaking it by switching it off is causal evidence, a notch harder than mere decodability. Evidence strength: one model, one architecture, and it does not transfer to other models for free.

The size of the difference is worth weighing. Far more can be read out of an internal state than the model's own pathway ever consults. Your body temperature is enough to read off the season, but body temperature is not the circuit that decides what you wear.

Rebuild English into a language no human could learn, retrain the model, and it struggles visibly more

There is a forceful claim going around: large models treat languages humans can learn and languages humans cannot exactly alike, so whatever they acquire has nothing to do with the human language capacity. Its virtue is that you can test it directly. In 2024 Kallini and colleagues built 15 "impossible languages" out of English, in three classes: word order scrambled by fixed rules; sentences reversed in whole or in part; verb number marking rewired into a rule that counts a few words ahead. Each was trained from scratch on the same GPT-2 architecture and compared against unaltered English as a control. All of them came out harder to learn, measured by perplexity (roughly, how well the model guesses the next word, lower being better).

The conclusion was squeezed from both sides at once. One side says the control was unfair: a 2025 paper argues specifically that no genuinely constituency-based language was among the comparisons. The other points out that the worst performer was full within-sentence scrambling, a "language" with almost no regularity to learn, so beating it proves little; and other work argues outright that probability and perplexity metrics cannot separate possible from impossible at all. Where things stand: the paradigm holds, the specific conclusion is still being fought over.

The two sides do not argue about one thing: the model is not a blank slate, it has a bias. Whether that bias is the human one has not been measured. Measuring it cleanly means feeding both sides comparable input, which is the thinking behind the BabyLM challenge — training data capped at 100 million words, roughly the order of what a child has heard by the age of thirteen. Today's large models have seen thousands of times that.

One last line that often gets collapsed into the others: batteries of minimal pairs, two sentences differing in one place where one is grammatical and one is not. BLiMP has 67 sets of a thousand pairs each, and human agreement with the labels overall is about 96 percent. The shape of the results is consistent: models are solid on items involving morphological agreement and visibly weaker on finer phenomena — for instance that English any is licensed only in certain negative or interrogative environments ("I didn't see any" works, "I saw any" does not). Which is why "did it learn grammar" is not a question one number can answer. It is dozens of separate curves.