This is the condensed version of the deep-dive essay of the same name. The deep dive has the full argument chain, every data source, and evidence grading; this version keeps the main line and says it plainly. The key numbers have been independently fact-checked (data as of July 2026).
AI writes code faster and faster, and the humans who check that code can't keep up — that problem is real. The industry's prescription is "buy another AI to check the code." That medicine genuinely works in narrow settings, but in exactly the places where you need it most, it's more like nesting dolls: an error-prone AI checking another error-prone AI — and the two are making increasingly similar errors.
First, is the demand real? AI has sped up the writing of code, but every line still has to be confirmed correct by someone before it ships — that step is called code review. Three unrelated data sources point the same way:
Fast to write, slow to check, traffic jammed at the checkpoint — the diagnosis holds. The problem is the prescription.
How do AI code review tools prove they work? They publish report cards. But those report cards follow one rule: whoever runs the exam comes first. By early 2026, at least six vendors had published their own evaluation benchmarks, and the publisher ranked #1 in every single one. One vendor's tool "catches 82% of bugs" on its own exam; a competitor re-tested it on the same five codebases and it scored 45; on a third company's exam, roughly 5 out of every 6 of its comments pointed at things that weren't real bugs at all. Same tool, three exams, three wildly different grades.
Then an independent research lab finally launched a leaderboard with no dog in the fight. Within about a month, three vendors each published a blog post declaring "we're #1" — one on the composite score, one on a single sub-metric, and one using a special lab-preview version to take first place (its actual shipping product ranked fourth).
The trick here was actually solved by science sixty years ago: saying "we catch X% of bugs" without saying "and here's how many false alarms we raise" is meaningless — turn any detector's sensitivity up and it will "catch more bugs," at the cost of drowning you in false alarms. And false alarms are the most expensive thing in code review, because they burn the one resource engineers are already short on: attention. Next time you see "we catch X% of bugs," ask one question: what's the false-alarm rate? If they won't say, grade it as a fail.
The really valuable data comes from companies that use AI code review on their own code and honestly publish the funnel.
Google: AI-suggested edits ended up resolving 7.5% of all human reviewer comments. Not 75% — 7.5%. And Google's paper admits that every metric they can measure is only an "easier-to-measure proxy" for productivity; how many real bugs the AI caught, nobody can measure.
Meta: their code-fixing AI scores 68 on their own curated exam; in real production, the share of its fixes that engineers actually accepted was 19.75%. Between the offline score and real-world value lies a full order of magnitude. The same paper has an even more telling experiment: Meta showed AI-generated fix suggestions to code reviewers — and reviews got 5.5% slower (statistically significant). The AI's output had itself become a new burden on the reviewer. The fix? Not a better model. They hid the AI's suggestions from reviewers.
Independent research fills in the other side: comments from open-source AI review tools led to actual code changes only 0.9% to 19.2% of the time, versus 60% for human review comments. And when someone ran four leading tools in parallel on the same 146 real PRs, 93.4% of the issues were flagged by exactly one tool, and zero issues were flagged by all four (the author works at one of the four vendors — noted — but the data is open-source and checkable). Four "inspectors" with almost no agreement on what counts as a problem: what they produce looks less like verification and more like opinions.
The deepest problem isn't that these tools aren't good enough yet. It's structural.
First, the AI inspector invents bugs. OpenAI trained a model specifically to critique code (CriticGPT). The positive results are real: its critiques were preferred over human contractors' in 63% of comparisons, and human-plus-AI teams reviewed more thoroughly than humans alone while hallucinating less than the AI alone. But in the same paper's abstract, the authors wrote one devastating sentence: the bugs the AI critic hallucinates "could mislead humans into making mistakes they might have otherwise avoided." The inspector is not a neutral filter — it injects new errors into the process.
Second, AIs are making increasingly similar mistakes. Using a second AI to check the first assumes their blind spots differ. But a 2025 study found the opposite trend: the more capable models get, the more similar their errors become. When the AI that writes the code and the AI that checks it share blind spots, stacking more layers of checking still misses the same bugs. That's the mathematically precise meaning of "nesting dolls": layer inside layer, same pattern painted on every one.
Third, counting on humans as the final gate is a plan that was sentenced to death in 1983. Forty years of human-factors research concluded: asking a person to watch an automated system that's right most of the time, just to catch its rare mistakes, gives "the human monitor an impossible task"; the complacency and over-trust that automation induces "cannot be prevented by training or instructions." A 2026 eye-tracking experiment confirmed it: tell reviewers "this code was written by an AI" and they stare at it longer — but review it no more thoroughly. And the most realistic picture comes from large-scale observation of AI-generated PRs: most AI-written PRs get no human review at all, and where "someone" does review them, that someone is usually another AI.
Flip the evidence over and AI code review genuinely pays off when three conditions hold at once:
Flip all three conditions — core business logic with no tests underneath, AI writing and AI reviewing, humans just clicking approve — and you have the standard portrait of the nesting dolls.
One honest footnote: in two years of boom, this industry has not run a single randomized controlled trial showing AI code review works — and not one named company has published a postmortem titled "why we turned our AI reviewer off." Rigorous evidence is missing in both directions.
If you're deciding whether to adopt it for your team: pick which repos go first by whether tests and type systems have your back, not by the vendor demo; in procurement, only accept catch-rate and false-alarm-rate as a pair — a one-sided number is a fail; during the pilot, sample the AI's comments yourself and count how many actually get acted on, then compare with the public numbers in this essay.
If you're the engineer being buried in AI comments: trusting everything and ignoring everything are both losing moves. The workable middle: triage by category — handle high-confidence classes (compile errors, clear API misuse) by default, bulk-downgrade the speculative classes (style opinions, "possible" concurrency issues), and spot-check the downgraded pile now and then. Two red lines: don't treat "tests pass" as proof of correctness, especially when the AI wrote the tests too; and don't let yourself become a reviewer who only reads AI summaries — the ability to read code with your own eyes is exactly the skill that holds its value in the AI era.
If you're building these tools: the first vendor to publish its false-alarm rate on its own benchmark page — with a note saying "this benchmark is run by a contestant" — will collect, for free, the trust the whole industry is currently bleeding.
The whole argument reduces to eight claims, ordered from hardest to softest evidence:
Worth watching: which way the first RCT points; whether the share of PRs merged with zero review keeps climbing or gets governed back; whether an independent evaluation protocol emerges that vendors can't harvest.