中文 →
Blog Deep-Read · 34 · gracewlindsay.com

Grace W. Lindsay
Every time a test of intelligence gets solved, it stops counting as a test of intelligence — thirteen years spent fighting a target that retreats

Grace W. Lindsay · Blog name Neurdiness (her own coinage: neuro + nerdiness) · Currently assistant professor of psychology and data science at New York University · Originally at neurdiness.wordpress.com, now 301-redirecting to gracewlindsay.com · First post 3 September 2012; the sitemap lists 36 posts, spanning twelve years and four months to the most recent one · Entirely free, no paywall; comments open · The output curve is wildly uneven, and that is itself information: 26 posts in the nine months from September 2012 to May 2013 (close to weekly, when she was a first-year PhD student), then a single post in the next four years (a 2015 podcast announcement), 5 posts across 2017–2018, and 4 since 2021 · Two clear inflection points: the cliff in May 2013 (she disappeared into her thesis work) and the complete change of register when she came back in 2017 — from science communication for a general audience to taking sides in arguments inside her own field · The blog is largely dormant: the most recent post is 24 January 2025. But she has not stopped writing; her main venue is now The Transmitter (most recent piece 13 October 2025) and the book she published in 2021 · This page is built on 16 blog posts read in full plus 5 pieces she published elsewhere under her own name, spanning 3 September 2012 to 13 October 2025: 9 of them from the earliest period (September 2012 to May 2013, inside the first 20 percent of the site's lifespan), with coverage on both sides of the 2013 cliff and the 2017 return · Selection was ordered primarily by what she herself puts forward on her Neurdiness Blog and My Writing Elsewhere pages (the author's own answer to "what should I read" outranks popularity), supplemented by internal backlinks (her longest 2018 post cites two of her own 2012 posts) and outside attention · Disclosures: this page has not read her 2021 book Models of the Mind, only her own announcement post and the essay she wrote about the experience of writing it, and passages touching the book are marked as such; for the criticism cited in section six, what this page read is the full abstract on the journal's page, not the target article itself, also marked

In 1739, Parisians queued up to see a duck that could digest.

The golden duck was the work of the inventor Jacques de Vaucanson. It stood on a pedestal and quacked. Offer it a pellet of feed and it would pick the pellet out of your hand and swallow it. Some time later it would excrete a gritty green waste from its back end. Vaucanson died in 1782 with his reputation as a pioneer of artificial digestion fully intact.

Sixty years on, the magician Jean-Eugène Robert-Houdin acquired the duck and set about repairing it. He took it apart and found that it had no digestive tract at all. The pellets fed in went into one container; pre-loaded, green-dyed breadcrumbs came out of another.

Grace Lindsay opened an essay with this duck in 2018. The essay asked whether modern artificial intelligence works the way a brain does, and how anyone could tell. The sentence she wrote next is the axle her whole body of thought turns on:

An intuitive response to hearing about Vaucanson's cheat is not to say that the duck is doing digestion differently but rather that it's not doing digestion at all. But a similar trend appears in AI. Checkers? Chess? Go? All were considered formidable tests of intelligence until they were solved by increasingly more complex algorithms. Learning how a magic trick works makes it no longer magic, and discovering how a test of intelligence can be solved makes it no longer a test of intelligence.Planes don't flap their wings: does AI work like a brain?, Aeon, 2018

That is not a quip, it is her root question. She wants to know whether "understanding the brain" is a target that automatically retreats the moment it is hit. She had been writing about exactly this thirteen years earlier. In December 2012, fresh from a conference marking Alan Turing's centenary, she wrote:

Somehow in the process of understanding thought, we seem to discredit any thought processes we can understand. […] We may miss the opportunity to explain thought because we are too afraid of explaining it away.Thoughts on thinking, 2012-12-17

Her method against the retreating target is both plain and brutal: go and dig up the sentence that put the target there in the first place, and measure against that.

In 2018 she wrote a Q&A of more than ten thousand words defending the use of convolutional neural networks as models of the visual system. The decisive move in it is not a technical argument, it is archaeology. She went back to find what vision scientists had written about what counts as a good model of the visual system before deep learning caught fire — and quoted Pinto et al. from 2007: progress on the brain's solution to object recognition requires building artificial recognition systems, and a working system is itself a particularly effective measure of success. Then she delivered the verdict:

From this perspective it's clear that CNNs do not represent a moving of the target in vision science, but more a reaching of it.Deep Convolutional Neural Networks as Models of the Visual System: Q&A, 2018-05-17

In the same piece she pulls off something even better. Everyone attacks these models by listing what they lack, so she wrote the most complete version of that list herself — spikes, saccades, separate excitatory and inhibitory cells, dynamics, feedback connections, skip connections, oscillations, dendrites, at which point the text inserts "[***inhale****]", and then continues: cortical layers, neuromodulators, a fovea, lateral connections, different cell types, binocularity, adaptation, noise, "and probably whatever your favorite detail of the brain is."

Having finished the list, she does not argue against it. She disqualifies it: no model is or should be a complete replica, and we cannot say a priori that any biological feature is necessary or that a model is a bad one for not having it. That is an empirical question.

Then she turns the whole audit around:

11. What do CNNs do that the visual system doesn't do? This, to me, is the more relevant question. Models that use some kind of non-biological magic to get around hard problems are more problematic than those that lack certain biological features.Ibid.

Missing something is not cheating; stealing something is. So she names the thefts one by one. Weight sharing: neurons at different locations in the same feature map use the exact same input weights, whereas in a real brain there is no "spooky action at a distance" keeping all those weights coordinated. Max-pooling: a unit's output equal to the single highest of its inputs, something real neurons can barely do — and the more suspicious part is that the operation started out as averaging, which neurons manage trivially, and was switched to taking the maximum purely because it performed better. That is the duck inspection: don't look at what comes out the back, look for the plumbing inside.

Eventually she turns the same ruler on her own trade. Neuroscientists use dimensionality reduction, decoding and tuning curves every day to read neural activity and then announce what the brain is doing. Nobody has ever tested whether those tools work. In 2022 she proposed a clean fix: use artificial neural networks as the calibration bench. In an artificial network you have the ground truth — every weight, every activation, every causal path is fully visible. Run neuroscience's analysis tools on it and see whether they recover what you already know is there.

Despite the fact that these tools shape the progress of the field as a whole, we have little empirical evidence that they are effective at quickly identifying the phenomena of interest.Testing the Tools of Systems Neuroscience on Artificial Neural Networks, arXiv, 2022

Notice that this is a complete inversion of the previous idea. In 2018 the artificial network is a model of the brain. In 2022 it is the calibration piece for neuroscience's instrument. She used the same object for two opposite purposes, and the reason was the same both times: the ruler we use to decide that we have understood something has itself never been measured.

In January 2025 she wrote a piece with nothing to do with neuroscience, using the same blade. The buzzword of the year was "agentic AI," and the second Trump administration had just opened. She put the two side by side: Silicon Valley racing to install agency in machines while politics raced to strip it out of people. She admits that she spent much of her life not feeling like an agent at all, only observing — she grew up in a cornfield-filled suburb "where nobody really did anything that mattered," so she never expected to either. Then her father died shortly before she finished her PhD, the pandemic arrived, and young readers started telling her that her book had changed their careers. She decided to take on the identity of an agent, and discovered immediately what it costs:

With agency comes responsibility. If you have the power to change the world then why aren't you? 'Can' quickly becomes 'ought'. […] If modern AI agents had to experience true agency, with the moral weight that is intrinsically tied to it, their chains of thought would snap under the pressure.2025: Agency gained and lost, 2025-01-24

This is the duck's last appearance. Something that looks like it is doing a thing is not necessarily doing that thing — a sentence she used in 2018 to defend a model, and in 2025 to convict one. The standard never moved. It is just that this time, the side that meets it is us.

Updates