With AI present the combo is stronger; once it's removed the human may be worse than before — keep two ledgers when evaluating AI. Never-failing automation is the most dangerous kind (33% vs 82% failure detection); experts follow wrong AI too, and a 20-hour AI course didn't protect (−14 points); unused skills rust, brain before hands. What works: experienced failure, outcome accountability, decision-time nudges, cheap verification. What doesn't: exhortations and lectures. Three unremovable floors: vigilance dies within half an hour, unknown failures can't be rehearsed, and the better the AI the worse the human's detection.
never-failing automation: 33% vs 82%20h AI course didn't help: −14ppeven experts fall to 45.5%after AI removed: 28.4%→22.4%
This is the condensed edition of the deep dive of the same name. Every key number went through graded verification (96 verbatim fidelity votes plus 9 contradiction-search and methods-audit verdicts — the audit withheld one widely quoted coefficient touched by a published correction, and drew boundaries around several popular conclusions). For the full argument, evidence grades, and sources, read the deep dive.
One doctor's detection rate, and one five-page old paper
In 2025, a top medical journal published a study that chilled the field: four hospitals introduced AI to help spot polyps during colonoscopies. Months later, nineteen physicians averaging 28 years of practice had their detection rate without AI fall from 28.4% to 22.4%. With AI present, the human-machine combo really was stronger — but this observational study's signal is that once the AI was taken away, the humans were worse than before. (Observational means confounds remain; we give it a full physical later.)
The script was written in 1983. A British psychologist predicted in five pages: the more advanced the automation, the more the human's job shrinks to "watch it, and take over when it breaks"; but skills rust without use, and nobody can truly keep watching a machine that almost never fails — the moment the system needs the human most is exactly the moment the human is least ready. She called it the irony of automation.
We ran this issue because every previous issue of this site ends at the same place: "the final call stays with a human." If the person making the final call gets softened by the AI itself, we've built our house on crumbling ground. So this issue turned over forty years of evidence: is the irony a death sentence, or a constraint you can engineer around?
The answer up front: it's a constraint, you can engineer around it, but three floors can't be removed — and every trick that works has a price tag.
Schematic: left = Hassan 2023 RCT meta (in-use gain, RR 1.24); right = Budzyń 2025 (senior physicians' non-AI colonoscopies, −6.0pp, observational, confounds unexcluded) — deployment evaluation must keep both books (Chapter 5)
The sharpest cuts from forty years of evidence
Automation that never fails is the most dangerous kind. The classic experiment: people multitasking while supervising an automated system caught 82% of its failures when it failed now and then — but only 33% when it had been constantly reliable. Not laziness: "it never fails" teaches your attention to leave. The most dangerous thing isn't bad AI; it's good AI that almost never errs.
When the machine is wrong, experts follow it too. Twenty-seven radiologists read mammograms: with correct AI advice, every experience group sat near 80% accuracy; with deliberately wrong advice, junior readers fell to 20% and even the most experienced group fell to 45%. A 2026 randomized trial cuts deeper: 44 physicians who had completed a 20-hour AI course still lost 14 percentage points of diagnostic accuracy when handed ChatGPT advice laced with errors. The course didn't protect them.
Skills rust fast — and the brain rusts before the hands. Meta-analysis: skills unused for over a year drop by more than a standard deviation, and cognitive skills decay faster than physical ones. Aviation measured it precisely: hand-flying precision tracks recent practice, not total career hours; and what actually degrades isn't stick-and-rudder — it's navigation reasoning and anomaly recognition. The judgment layer. Bad news: judgment is exactly what AI automates.
Aviation played the whole script out. Air France 447: the autopilot handed a perfectly flyable plane back to three pilots; the stall warning sounded continuously for 54 seconds; nobody mentioned it once; 228 people died. The report's verdict: a safety model that uses the human as automation's backup enters "common failure mode" — losing control makes the situation unreadable, and unreadability deepens the loss of control. US regulators reviewed 26 accidents: over 60% involved manual handling errors; in over half, pilots were "out of the control loop and not prepared to assume control." But say the other half too: aviation got dramatically safer in the automation era — the fatal accident rate fell 60% in two decades. Automation saves lives in aggregate, while concentrating the risk at the moment of handback.
Schematic: Parasuraman, Molloy & Singh 1993 (students, MATB multitask platform) — automation that never fails teaches attention to leave; lab failure rates far exceed real systems, so extrapolate with care (Chapter 3)
But don't convict the human just yet
Two famous numbers turn out to wrong the human:
"Doctors ignore 90%+ of system alerts" — in fact, when experts reviewed those overrides, they agreed with the doctor 95.6% of the time. The problem is alarm systems that cry wolf, not derelict humans. Audit the system's signal quality before auditing the human.
"Studies prove AI makes you dumber" — most of the viral 2024-25 studies fail methods review: the MIT EEG study is an unreviewed preprint with 54 participants (an independent analysis says the design needed ~159), and the authors themselves beg the media not to use the word "dumber." Last wave's star finding — "Google ruins your memory" — later failed replication twice. Don't trust the panic; the trustworthy alarms (the cuts above) happen to point the same way anyway.
What works, and what it costs
Forty years of intervention evidence, scorecard form — every row has a price:
Letting people experience AI failure works. Telling trainees "AI can make mistakes" does nothing; letting them get genuinely burned once in a drill, and catch it, measurably cuts complacency. (That 20-hour course was the former.)
Accountability works. Make it explicit that the final outcome is on you, and both blind trust and missed errors drop.
Forcing thought before answers partly works. Making people commit to their own judgment before seeing the AI's cuts "following the AI into error" — the price is that people find it more taxing, and one independent experiment found it fails against bias-type errors. Friction is not a panacea; design matters.
Making verification cheap works. People skip checking the AI mostly because checking costs as much as redoing. Design AI output so it can be verified at a glance, and people actually verify. Giving the decision-maker a verification path far cheaper than redoing the work is the single most valuable design move.
A regular "manual dose" works — but often never happens. Doing the task periodically without AI preserves the skill. Aviation's lesson: regulators urged more hand-flying, and an audit three years later found nobody ever checked whether airlines complied. Writing the prescription is useless; watch the medicine get taken.
Three floors you can't remove: humans can't keep watching something that almost never fails for more than about half an hour — that's physiology; drills only rehearse known failures, and automation's new failures are by definition the unimagined ones; and the better the AI, the rarer its failures — so human detection gets worse and false alarms dominate. The veto seat's value and its reliability erode each other by construction.
So when you design a "human makes the final call" seat into an AI workflow, ask four questions: How many real anomalies does this seat see per day (too few → no full-time watcher; use spot checks plus automatic backstops)? How much cheaper is verifying one AI output than redoing it? When did this person last do the job without AI? And: when the AI breaks, does it break visibly? That last one is the hardest and most valuable in the LLM era, because LLMs fail fluently.
Schematic: effect sizes, boundary conditions, and costs per lever in Chapter 8; 'effective' is not universal — most effects are design-dependent
How to check this essay's judgments
The deep dive ends with eleven testable claims, ranked by evidence strength. In plain words:
Wrong AI advice drags people down — this is nailed down (a randomized trial plus multiple independent teams; physicians dropped 14 to 69 points); seniority softens the fall but doesn't prevent it.
"Stronger while using it" and "weaker after it's gone" are two separate ledgers: the first has dozens of randomized trials; the second has exactly one observational study so far. Keep both books before deploying.
Almost-never-wrong automation is the most dangerous regime — the 33%-vs-82% experiment; don't rely on bad automation, and fight complacency with good automation.
Unused skills rust, and the brain rusts faster than the hands; freshness comes from recent practice, not career totals.
The 1983 prophecy's report card mostly reads "confirmed": skill decay, the vigilance ceiling, and "least ready when most needed" all have empirical support; only "humans can't real-time-check a stronger system" remains at the reasoning stage.
Forty years grew no "the prophecy was wrong" school — only a "how to mitigate" literature; the 40th-anniversary verdict: it applies to AI even more.
Generic courses alone don't work; what works is experienced failure, outcome accountability, decision-time nudges, and cheap verification — each with measured effects, and each with a price or boundary.
"Ignoring alerts" is usually right (95.6% expert-endorsed) — fix the system before blaming the person.
Inserting a human into the loop does not equal safety: undesigned human review is a compliance placebo; design the human's role, then test it.
Most viral "AI rots your brain" studies are methodologically unsound, but the solidest new evidence does point the bad way — distrust the panic, trust the alarm.
Nobody has ever measured this in professional programmers: no published before/after skill study exists for working developers — the most overdue experiment in the field.
What to watch: whether the endoscopy finding gets a prospective replication and holds; who first measures working developers' skill retention; and when "fails visibly" design for LLM workflows gets its first measured data.
The things that matter most
Starting today, evaluate AI deployments with two ledgers: joint performance with AI present, and what's left of the human when AI is absent. Nobody kept the second ledger before 2025 — yet it decides what you hold on outage day, on out-of-scope tasks, and at the next migration.
Don't staff a "watch the AI full-time" seat — that's a bet against forty years of vigilance research. Use spot checks, automatic backstops, and a scheduled "hands-on without AI" dose — and audit that the dose actually happens.
Spend the training budget not on "AI can make mistakes" slides but on drills where people genuinely get burned once — or cheaper still, a nudge at the decision moment. Both beat courses.
Every AI output that reaches the decision-maker needs a verification path far cheaper than redoing the work; if you can't provide one, the seat is furniture.
And keep the two-sided sentence: automation saves lives in aggregate, while concentrating risk at the moment it hands back to the human. Both halves are true; either one alone is a lie.