模型的「意外」能预测人的眼动

「一把」两个字出来，你已经把大半本词典划掉了

1981 年 Ehrlich 和 Rayner 做了一件很直接的事：同一个目标词，放进两种上下文——一种前文把它逼得几乎只能是它，一种前文什么也没框住。结果是，高约束那一版里读者更常整个跳过这个词；就算停了，停留时间也更短。他们还偷偷把目标词拼错了一部分，发现上下文越紧，读者越不容易发现那个错——你看到的一部分是你自己填进去的。

这说明预测不是读完一句话之后的补救，是边读边发生的，而且在词进入视野之前就已经开始。

汉语把这件事做得特别显眼，靠的是量词。你说「一把」，后面的名词就跳进「有把手的那一类」——伞、刀、椅子；说「一只」，是动物和成对物品里的一个——猫、鞋；说「一条」，是细长的——鱼、路、腿。英语的 a 什么也不告诉你。这不只是语法书上的说法：脑电实验里，一个和后面名词搭不上的量词，在读到量词那一刻就会引出 N400（一个在词义对不上时出现的脑电反应），而且这个反应是分级的——量词离读者预期的那个词越近，反应越小。2023 年还有一组用表征相似度分析做的结果：名词出现之前，「活物类」量词和「非活物类」量词引出的脑活动模式就已经分开。

所以押注是真的在押。下一个问题很自然：这个注押得多准，能不能变成一个数？

EN In 1981 Ehrlich and Rayner did something very direct: they placed the same target word into two kinds of context — one in which the preceding text pinned it down almost completely, and one in which nothing constrained it. Readers skipped the word entirely more often in the high-constraint version, and when they did fixate it, the fixation was shorter. The researchers also quietly misspelled part of the target word, and found that the tighter the context, the less likely readers were to notice the error: part of what you see, you supply yourself. Prediction, then, is not repair work done after a sentence ends. It runs while you read, and it starts before the word reaches your eyes. Mandarin makes this unusually visible through its classifiers. Say 一把 (yī bǎ, "one handle-shaped-thing") and the following noun jumps into the class of objects with handles — 伞 (sǎn, "umbrella"), 刀 (dāo, "knife"), 椅子 (yǐzi, "chair"). Say 一只 (yī zhī) and you land among animals and one-of-a-pair items — 猫 (māo, "cat"), 鞋 (xié, "shoe"). Say 一条 (yī tiáo) and it is long thin things — 鱼 (yú, "fish"), 路 (lù, "road"), 腿 (tuǐ, "leg"). The English article "a" tells you none of this. Nor is it merely a claim from grammar books: in event-related potential studies, a classifier that clashes with the upcoming noun elicits an N400 — the brain response that shows up when a word's meaning does not fit — at the moment the classifier itself is read, and the response is graded, growing smaller as the classifier moves closer to the word the reader expected. A 2023 study using representational similarity analysis found that patterns of brain activity already diverge before the noun appears, depending on whether the classifier constrains toward animate or inanimate nouns.

汇率是：每把概率砍一半，就多花固定的几毫秒

2001 年 Hale、2008 年 Levy 各自把这件事定成一个量：给定前文，算出这个词出现的概率 p，取 −log p，叫它意外度（surprisal）。概率越低，这个数越大；概率是 1 的时候它是 0。取 2 为底的话，单位就是比特。

关键不在这个定义，在它跟毫秒之间是什么形状的关系。2013 年 Smith 和 Levy 把概率从接近 1 一直推到小于百万分之一，横跨六个数量级去看阅读时间怎么变，结论是：关系是对数的，不是线性的。翻成人话——从「十拿九稳」掉到「五五开」要多花的时间，和从千分之一掉到两千分之一要多花的时间，是一样多的。每把概率砍一半，就加上固定的一小笔。这一笔大概值多少？自然阅读的估计在每比特两毫秒上下，自定步速阅读里大概四毫秒。一个比特，几毫秒，不多，但它稳定到能从噪声里捞出来。

2024 年 Shain 等人把这件事推到更大规模：多个眼动与自定步速阅读语料、用现代语言模型重估概率，结论仍是对数形状胜过线性形状。2023 年 Wilcox 等人换了个方向验证——十一门语言、分属五个语系（韩语、土耳其语、希伯来语、芬兰语，加上荷、英、德、希腊、意、俄、西），用多语眼动语料 MECO，三条预测（意外度能预测阅读时间、预期中的意外度也能、这条关系是线性于意外度的）都站住了。证据强度在这里可以给得比较硬：多语料、跨语系、独立课题组。

EN In 2001 Hale and in 2008 Levy each turned this into a quantity: given the preceding context, compute the probability p of the word that actually appears, and take minus its logarithm. Call it surprisal. The lower the probability, the larger the number; when the probability is 1, it is 0. Take logarithms base two and the unit is the bit. What matters is not the definition but the shape of its relation to milliseconds. In 2013 Smith and Levy pushed probability from near one down below one in a million — six orders of magnitude — and asked how reading time changed. The relation is logarithmic, not linear. In plain terms: the extra time it costs to fall from "nearly certain" to "a coin flip" is the same as the extra time it costs to fall from one in a thousand to one in two thousand. Every halving of probability adds the same fixed increment. How large is that increment? Around two milliseconds per bit in naturalistic reading, roughly four in self-paced reading. A few milliseconds per bit is small, but stable enough to be pulled out of the noise. In 2024 Shain and colleagues scaled the question up across several eyetracking and self-paced reading corpora with probabilities re-estimated from modern language models, and the logarithmic shape still beat the linear one. In 2023 Wilcox and colleagues tested it from another direction: eleven languages across five families — Korean, Turkish, Hebrew, Finnish, plus Dutch, English, German, Greek, Italian, Russian and Spanish — using the multilingual eyetracking corpus MECO. All three predictions held. The evidence here can be labelled fairly strong: multiple corpora, multiple language families, independent groups.

模型越好越像人——一直到它好过头

既然要一个概率，就得有个东西来给。衡量一个语言模型猜词猜得好不好，常用的数叫困惑度（perplexity）：可以粗略读成「平均而言，它像是在几个等可能的选项里挑一个」——数字越小，猜得越准。

2018 年 Goodkind 和 Bicknell 报告了一个很干净的规律：模型的困惑度越低，它给出的意外度对人类阅读时间的解释力就越强，而且两者近似成一条直线。当时很自然的推论是，把模型继续做大做强，就会越来越像人。

2023 年 Oh 和 Schuler 把这条线延长，发现它拐了。在同一个模型家族内部（他们测了 GPT-Neo 和 OPT 两个系列，用了两份阅读语料），参数更多、困惑度更低的那些模型，拟合人类阅读时间反而更差，而且是严格单调的——困惑度和拟合优度之间是一条稳定的对数线性关系，只是方向跟人们期待的相反。同年还有一个更具体的落点：Transformer 模型的意外度在训练量大约二十亿词左右时最像人，再多就开始偏。

解释的方向是训练数据量：见得太多之后，模型对生僻人名和罕见搭配也不再意外了，而人还在那儿卡住。它猜得太好，好到不再是人的那种猜法。

再往外一步更不整齐。2021 年 Kuribayashi 等人用日语眼动语料做同样的比较，结论是「困惑度越低越像人」在日语上不成立。日语谓语在最后，信息铺开的方式和英语不同；是不是语序造成的还没有定论。证据强度要标弱：单语种、单语料。

EN If you need a probability, something has to supply it. The usual measure of how well a language model guesses the next word is perplexity, which can be read loosely as "on average, it is picking from about this many equally likely options" — the smaller the number, the better the guess. In 2018 Goodkind and Bicknell reported a clean regularity: the lower a model's perplexity, the better its surprisal explained human reading times, and the two traded off along an approximately straight line. The natural inference at the time was that making models bigger and stronger would make them steadily more humanlike. In 2023 Oh and Schuler extended that line and found it bends. Within a single model family — they tested the GPT-Neo and OPT series against two reading corpora — the models with more parameters and lower perplexity fit human reading times worse, and strictly monotonically so: perplexity and goodness of fit stand in a stable log-linear relation, just running the opposite way from what people expected. The same year brought a sharper landmark: Transformer surprisal is most humanlike at around two billion training tokens, and drifts off beyond that. The favoured explanation is sheer training volume. Having seen too much, a model is no longer surprised by rare personal names, technical terms or unusual collocations — while people still stumble there. It guesses too well to guess the way a person does. One step further out, things get messier still. In 2021 Kuribayashi and colleagues ran the same comparison on a Japanese eyetracking corpus and found that "lower perplexity is more humanlike" does not hold in Japanese. Japanese puts its predicate last, so information is spread through a sentence differently than in English; whether word order is the cause remains unsettled. The evidence at this layer deserves a weaker label: one language, one corpus.

它说得准「哪儿会卡」，说不准「卡多久」

英语里有一句著名的句子：The horse raced past the barn fell. 读到 raced 的时候你几乎一定把它当成主句的谓语——「那匹马跑过了谷仓」；一路顺到 fell，整句塌了，你得回头把 raced past the barn 重新读成修饰语，整句其实是「那匹被赶过谷仓的马摔倒了」。同类的还有 While the man hunted the deer ran into the woods：the deer 先被当成 hunted 的宾语，读到 ran 才发现它是后半句的主语。这类句子叫花园幽径句——沿着一条看着好走的小路进去，走到底发现得原路退回来。

意外度在这里表现得很对：它确实在 fell、在 ran 上给出高值，也就是准确地指出了「人会在这儿卡」。问题出在量上。2021 年 van Schijndel 和 Linzen 拿三类花园幽径结构做检验，结果是：意外度严重低估了人真正卡住的幅度，而且低估的倍数因结构而异——这一条更要命，因为它意味着不存在任何一个「每比特值多少毫秒」的汇率能同时对上三类。2024 年 Huang 等人做了一个更大的基准，结论一样，还多出一条：连同一类结构里不同句子之间的差异，意外度也预测不了。

合理的读法是：阅读时间里至少有两笔账。一笔是预期落空，意外度管这一笔，管得相当好；另一笔是把已经搭好的结构拆掉重搭，意外度完全不涉及。前者解释「哪里」，后者解释「多久」。

那么这个相关到底说明了什么？说明人对语言里的统计规律敏感，而且敏感的方式是对数的、在线的、跨语系稳定的。它不说明脑子里跑着一个 Transformer——模型在这套理论里只是概率的估计器，估得准不准，和「内部像不像人脑」是两件事。「模型越好越像人，一直到它好过头」那节的反转正好把这两件事分开了：如果拟合度衡量的是像不像脑，模型越强就该越拟合，而实测是反的。

EN English has a famous sentence: "The horse raced past the barn fell." Reading "raced", you almost certainly take it as the main verb — the horse ran past the barn — and sail on until "fell" collapses the whole thing, forcing you back to reread "raced past the barn" as a modifier: the horse that was raced past the barn fell. Another of the same kind: "While the man hunted the deer ran into the woods", where you first take "the deer" as the object of "hunted" and only discover at "ran" that it is the subject of the next clause. These are garden-path sentences: you walk down an inviting path and reach the end only to find you must back out the way you came. Surprisal performs well here in one respect. It does assign high values at "fell" and at "ran", pinpointing exactly where a reader will stall. The trouble is the magnitude. In 2021 van Schijndel and Linzen tested three classes of garden-path construction and found that surprisal drastically underpredicts how badly people actually stall — and that the size of the shortfall differs across constructions, which is the more damaging result, because it means no single exchange rate of milliseconds per bit can fit all three at once. In 2024 Huang and colleagues ran a much larger benchmark to the same conclusion, adding that surprisal also fails to predict variation among individual sentences within one construction. The sensible reading is that reading time carries at least two separate bills. One is the cost of a violated expectation, which surprisal handles, and handles well. The other is the cost of tearing down a structure already built and rebuilding it, which surprisal does not touch at all. The first explains where; the second explains how much. So what does the correlation actually show? It shows that people are sensitive to the statistics of their language, and that the sensitivity is logarithmic, incremental, and stable across language families. It does not show that a Transformer runs inside the head. Within this theory a model is only an estimator of probabilities, and how good an estimator it is stands apart from whether its internals resemble a brain. The reversal described in the section on models that guess too well separates those two things precisely: if goodness of fit measured brain-likeness, stronger models would fit better, and measurement says otherwise.