Data as of July 2026. The 15 load-bearing claims in this essay (benchmark figures, production funnels, reversal findings) were each challenged by 3 independent verifiers (downloading primary sources, checking numbers and definitions word for word, searching for corrections and counter-evidence): 9 passed unchanged, 6 were amended per verifier findings (mostly definitional tightening — e.g., what human+AI teams actually gain in CriticGPT, and the pr-review-bench author's conflict of interest), 0 were overturned. Six further figures (METR, Sonar, Google migrations, DORA, Faros 2025, Anthropic sessions) carry over from When Code Becomes Cheap with their locked wording. Numbers from vendors and interested parties are flagged as such throughout.
When Code Becomes Cheap concluded: AI pushes the cost of writing software toward zero and pushes the cost of confirming it is right into the organizational bottleneck position. Since then, the industry has produced an answer almost too convenient: verification too slow? Buy another AI to do the verification. By mid-2026 that answer is an industry — more than one in five code reviews on GitHub is performed by Copilot (60 million cumulative, platform self-report); Cursor, a company that writes code, acquired Graphite, a company that reviews it, with CEO Michael Truell's stated reason being "The way engineering teams review code is increasingly becoming a bottleneck to them moving even faster as AI has been deployed more broadly within engineering teams"; benchmarks independent and otherwise, funding rounds, and "#1" claims have piled up dense enough to trip over each other.
But the answer skips a question. Code review became the bottleneck because LLM output is unreliable and needs an independent check — and the entity now hired to perform that check is another LLM, a generative model with the same hallucinations and the same absence of correctness guarantees. Has the bottleneck been solved, or nested: AI writes the code, AI reviews the code, humans review the AI's review — and then what?
That is not a rhetorical question; it is an empirical one. The plan: first confirm the demand side (is the verification bottleneck real?); then dissect the supply side's evidence (vendor benchmarks); then examine three harder bodies of evidence — first-party production data from big tech, independent academic studies, and the foundational literature on whether an LLM can be a verifier at all; finally, a verdict: under what conditions AI code review is a cure, and under what conditions it is turtles all the way down.
First rule out one possibility: if the "verification bottleneck" were itself a marketing invention, "cure or turtles" would be moot. The evidence does not permit that reading.
The micro time ledger. METR's randomized controlled trial (16 experienced open-source maintainers, 246 real tasks, July 2025): being allowed AI made tasks take on average 19% longer (figure-read 95% CI roughly +2% to +40%), while developers predicted they'd be 24% faster beforehand and still believed they'd been ~20% faster afterward; screen-recording annotation showed roughly 9% of working time spent reviewing and cleaning up AI output (74 recordings, 84 hours, a subset covering ~29% of total work time). This must be bundled with METR's February 2026 update: a re-test of the original cohort showed a point estimate of ~18% speedup but with a CI of (−38%, +9%) crossing zero, a newly recruited 47-person cohort showed roughly no effect (−4%, CI −15% to +9%, skewed toward greenfield small repos), and METR called the new data an unreliable signal and said it is working on changes to the study design. The 19% point estimate is no longer durable; what is durable is the ~40-percentage-point gap between perception and measurement, and the structural shift of time from writing to reviewing.
Organizational telemetry. Faros 2025 data (10,000+ developers, vendor telemetry): high-AI-adoption teams merged 98% more PRs at 154% larger size with review time up 91%, and cycle time uncorrelated with adoption — the throughput gain queues up at review. The April 2026 sequel, The Acceleration Whiplash (22,000 developers, 4,000+ teams, comparing each organization's lowest vs highest AI-adoption periods; again vendor telemetry, not a controlled experiment): task completion per developer +33.7%, but median PR review time +441.5%, incidents per PR +242.7%, and PRs merged without any review at all — human or agentic — up 31.3%. The report's own reading: "The more likely explanation is that reviewers cannot keep pace with the volume of AI-generated code arriving for their attention." LinearB's telemetry across 8.1 million PRs (2026): AI-assisted PRs wait 4.6× longer for first review and merge at under half the rate of human-written code. The three telemetry sources disagree wildly on magnitude and agree on direction.
The supply flood. GitHub Octoverse 2025 (platform telemetry): 43.2 million PRs merged per month, up 23% year over year; Copilot coding agent produced over 1 million PRs within five months of launch. Anthropic self-reports that as of May 2026, over 80% of code merged into its codebase was authored by Claude (vendor figure, includes supervised agent output). The denominator is exploding while the numerator — human review attention — is fixed. That is the scissors.
The organizational smoking gun. Google's internal migration paper (ICSE-SEIP 2025, first-party): in the JUnit3→JUnit4 migration, ~87% of AI-generated code was committed without modification, and — verbatim — "The bottleneck in the process was the speed at which engineers could review the changes. We purposefully limited the number of changes we generate every week to avoid overwhelming reviewers." Generation overcapacity, verification rate-limited: the diagnosis from When Code Becomes Cheap needs no re-arguing here. The new question is: does handing verification to an AI actually work?
How does the industry demonstrate that AI review works? Mostly with benchmark scores. Lay the 2025-2026 public benchmarks side by side and one structural fact emerges.
As of February 2026, at least six vendors had published their own AI code review benchmarks, and the publisher ranked first on its own benchmark in every single case: Greptile (82% bug catch rate), Augment (59% F1), Tenki (41.7 F1), Entelligence (47.2% F1), Qodo's own benchmark (60.1% F1), DeepSource's security benchmark (100% precision). The article that nailed this to the wall is literally titled "Every AI code review vendor benchmarks itself, and wins" — written by DeepSource, itself a contestant; the critique of the game is also a move in the game.
The same tool's score varies by a factor of 3.7 across benchmarks. Greptile catches 82% of bugs on its own benchmark (July 2025; 50 real bug-fix PRs across 5 large OSS repos); Augment, re-testing on the same five repos with its own expanded and revised dataset, scores Greptile at 45% (F-score); on Tenki's benchmark (May 2026, 122 real production bugs), Greptile shows 36.1% recall and 15.9% precision. The three sets of figures use three different metrics — which is itself part of the problem: no two vendors' headline numbers share a unit of measure.
Martian (a San Francisco research lab staffed by ex-DeepMind/Anthropic/Meta researchers) runs the Code Review Bench, the closest thing to an independent leaderboard: 200,000–300,000 real PRs tracked, measuring whether developers actually act on comments, updated daily. Yet within roughly four weeks in March 2026, three vendors each published a "we're #1" post: CodeRabbit claimed #1 in F1 (51.2%, March 3); Qodo claimed #1 overall (64.3% F1 — but for a research-preview configuration called Qodo Extended, while its shipping product ranked #4 on the same board at 47.9% F1; March 15); Baz claimed #1 in precision (F0.5, based on the February 26, 2026 snapshot; March 29). A daily-updated, multi-metric leaderboard hands every vendor a timestamp-and-metric combination on which it is truthfully-ish first. CodeRabbit itself wrote this section's most honest sentence: "AI code review benchmarks have mostly been published by other code review vendors (whose tools always seem to come out on top in their benchmarks)."
This melee doesn't need case-by-case refereeing, because as a whole it replays an error signal detection theory (Green & Swets, 1966) resolved sixty years ago. SDT's core contribution is separating sensitivity (the ability to distinguish signal from noise) from response bias (how loose the alarm threshold is): hit rate and false-alarm rate must be reported as a pair, because a hit rate alone — in the textbook formulation of Stanislaw & Todorov (1999) — leaves it "not clear whether the conditions differ in sensitivity, response bias, or both." Loosen the threshold and any detector's "catch rate" goes up — at the cost of exploding false alarms, and false alarms are precisely the most expensive commodity in code review, because they consume the already-scarce resource of human attention.
Hold that ruler up: Greptile's benchmark page publishes only catch rate (recall) — no precision, no false-positive accounting — while under Tenki's methodology its precision is 15.9%, i.e., roughly 5 of every 6 comments do not point at a real bug. Graphite is the mirror image: 6% catch rate on Greptile's benchmark (last place), 66.7% precision on Entelligence's (best in field) — which tool "wins" depends entirely on which axis the benchmark's owner monetizes. And all of these benchmarks share a set of methodological weaknesses: tiny samples of 50–122 bugs (SWR-Bench measured that merely aggregating multiple review runs lifts F1 by up to 43.67% — run-to-run variance alone can reorder such a leaderboard), synthetically injected bugs (Qodo injected 580 defects into real PRs, with LLM-generated ground truth), and near-universal LLM judging — AI grading a contest of AI reviewing AI-written code. The turtles start in the eval layer.
The academic verdict is blunter: a February 2026 survey covering 99 papers concludes that existing code review datasets "are scattered, vary widely in design, and provide limited insight into what review capabilities are actually being assessed." The ruler for measuring AI review has not itself passed metrology.
Beyond vendor benchmarks, three companies have published first-party data from at-scale production deployments. Their value is not prettier numbers — it is that they honestly expose the same shape: an order-of-magnitude funnel between offline evaluation and production reality.
Google's ICSE-SEIP 2024 paper (peer-reviewed, first-party) reports: in production, code-change authors resolve 7.5% of all human reviewer comments by applying an ML-suggested edit (4.9% in the V1 release). The full funnel: 100% of comments come in, 49% get a confident model prediction, 10.7% get previewed by the author, 7.5% get applied. The often-misread "50%" is the target precision to which suggestion confidence was calibrated (tuned so about half of suggested edits are correct — later relaxed to 40%, which improved downstream acceptance), not a coverage goal of resolving half of comments.
The most quotable thing in the paper is not 7.5% but the authors' own methodological confession: all these forms of evaluation "act as easier-to-measure proxies" of developer productivity. Nowhere does the paper measure how many real bugs were caught — not an oversight, but because that quantity is nearly unmeasurable in production. The largest, most serious deployer on record openly admits to navigating by proxy metrics.
Meta's MetaMateCR paper (arXiv 2507.13499, first-party) supplies this essay's sharpest number pair: the fine-tuned LargeLSFT model scores 67.96% exact-match offline (GPT-4o: 59.22%); in production, the ActionableToApplied rate — the share of actionable-classified comments whose AI patch the author actually lands — is 19.75%. The denominators differ and must be stated: offline is 206 hand-curated high-quality eval points; production is the full comment stream filtered first by an LLM actionability classifier (GPT-4o), then by UX display, then by a human decision. It is precisely because three funnel layers sit in between that the pair is informative — the offline score is a grade on a self-curated exam; the production apply rate is the market price. The intermediate metric is just as cold: of AI patches actually shown to authors, fewer than three in ten are applied (ShownToApplied ≈ 28.74%) — and swapping in a stronger model produced no statistically significant change. The bottleneck is not model quality; it is the human judgment at the end of the funnel.
The same paper buries an A/B experiment that may be the most direct first-party evidence for the turtles worry: Meta initially showed AI-generated patches to both authors and reviewers — and both the time diffs spent in review and the time reviewers spent looking at diffs showed statistically significant regressions (+5.5% each, p=.029). The AI's output had itself become new verification load on the reviewer. The fix was not a better model but a product change: collapse the AI patch by default when a reviewer views the diff (authors still see it), after which no metric showed a significant regression (p=.86/.67/.67). Meta experimentally measured "AI assistance slows reviewers down" — and fixed it by hiding the AI.
ByteDance's BitsAI-CR (first-party, arXiv): architecturally, a RuleChecker generates candidate issues and a ReviewFilter then re-validates them to cut false positives — a vendor voting with its architecture: raw LLM review output must pass a noise-reduction layer before humans may see it. Self-reported precision is 75%, and ByteDance invented an Outdated Rate metric (26.7% for Go: the share of comments whose target lines get changed in subsequent commits) to proxy "the comment actually mattered" — the invention of a new metric being itself a first-party admission that acceptance-style metrics are unreliable. Amazon's CodeGuru data from ECOOP 2022 provides the pre-LLM control pole: 8 narrow, precise static rules, 83% recommendation acceptance — the narrower the rule, the higher the precision and the acceptance; LLM review runs the opposite, broad-coverage bet.
Independent industrial research supplies the counterexample: an ICSE 2025 SEIP deployment study (3 commercial projects, 4,335 PRs) found 73.8% of automated comments were marked resolved — while mean PR closure time rose from 5h52m to 8h20m, alongside "faulty reviews, unnecessary corrections, and irrelevant comments." "Comments resolved" and "shipping faster" can point in opposite directions: the former is activity, only the latter is a result.
Step outside vendor and big-tech self-reporting and the independent picture is colder.
Addressing rates: an order-of-magnitude gap. A study of 16 open-source GitHub-Actions-based AI review tools (178 repos, 22,000+ comments, 2026 revision): among comments judged valid, the share of AI comments leading to a code change ranges 0.9%–19.2% by tool (file-level tools 0.9%–4.2%, hunk-level 6.5%–19.2%), versus roughly 60% for human comments under the same rubric; 87.0% of file-level AI comments are followed by no code change at all (human comments: 8.2%). The caveat travels with the number: these are free open-source Actions, not Copilot or CodeRabbit's SaaS — but an order-of-magnitude gap is unlikely to be explained by product tier alone.
Unsupervised review agents: a 23-point merge penalty. A study on the AIDev dataset (3,109 PRs) directly tested the industry claim that code review agents can handle 80% of open-source PRs without human involvement: PRs reviewed only by CRAs merge at 45.20%, 23.17 percentage points below human-only PRs (68.37%), with significantly higher abandonment; 12 of 13 CRAs tested show average signal ratios below 60%. A second study on the same dataset draws the more structural picture: most AI-generated PRs receive no human review at all, and where review happens, agents — not humans — dominate. AI writes, AI reviews, humans absent: the turtles are not a thought experiment; they are an observed fact in the AIDev dataset (observational, with author-declared selection bias from agent-generated PRs).
The tools don't even agree with each other. A practitioner test (May 2026; CodeRabbit, Greptile, Sentry Seer, and Cursor BugBot run in parallel on 146 real merged PRs for three and a half weeks; data open-sourced; the author discloses employment at Sentry — conflict noted): of 679 findings deduplicated by issue, 93.4% were caught by exactly one tool, and zero findings were flagged by all four. Four "verifiers" with almost no intersection on what counts as a problem — which both caps any single tool's plausible recall and suggests these outputs are sampled opinions rather than verification verdicts.
Evidence gaps, on the record. As of July 2026, randomized controlled trials or randomized field experiments of AI code review tools: none published. The industry generates billions of AI comments monthly while the top tier of effectiveness evidence remains observational telemetry and before/after comparisons. A second gap deserves the same candor: no named company has published a first-hand "we turned our AI reviewer off, and here's why" engineering post — shutdown evidence exists only as vendor hearsay and anonymous forum accounts. The absence is informative: this market currently has neither rigorous proof of effectiveness nor a rigorous failure postmortem.
The above is "how AI review is going." One level deeper: does it work in principle? The founding assumption of the scalable-oversight program (Leike et al. 2018 on reward modeling; Leike himself in a 2023 interview, using exactly the example that verifying a suggested bug is much easier than finding all bugs) is that verification is easier than generation, so weaker verification can leverage stronger generation. The 2024-2026 literature attaches four footnotes.
Footnote one: machine verifiers themselves wave through large volumes of errors. The numbers in "Rethinking Verification for LLM Code Generation" (arXiv 2507.06920) need precise framing: it measures synthesized test suites as verifiers, with Verifier Accuracy defined as the test suite rejecting every known wrong solution for a problem. The best method, SAGA, achieves 32.58%; the baseline, 16.72%; and the human-maintained LiveCodeBench-v6 scores just 19.61% on a shared subset. This number must not be read as "LLM review is 32.58% accurate" — it says something more basic: even for running tests, the hardest form of verification available, the best current automated construction fully backstops only about one problem in three. A far softer verification form — "an LLM reads the diff and offers opinions" — has even less claim to being treated as a reliable verification layer by default.
Footnote two: generation and verification disagree inside the same model. The GV-consistency study (ICLR 2024): GPT-4's generator and validator sides agree only 76% of the time about its own just-produced output. The Stechly–Valmeekam–Kambhampati experiments: GPT-4 self-critique causes significant performance collapse on reasoning and planning tasks, while a sound external verifier yields significant gains — the authors directly reject the complexity-theoretic intuition that verification is easier than generation as applied to LLMs, which perform approximate retrieval, not formal reasoning.
Footnote three: CriticGPT — the most honest specimen; cite both faces. OpenAI's CriticGPT (arXiv 2407.00215) is the most serious first-party study of the AI-reviews-AI program, and its positive results are real: on code with naturally occurring LLM errors, model-written critiques were preferred over human contractors' critiques by (blinded) human annotators in 63% of cases; Human+CriticGPT teams wrote critiques substantially more comprehensive than humans alone, with hallucination and nitpick rates well below the model alone — the team crossed the model-only Pareto frontier. But the verifier-amended full reading has three more sentences: the teams' comprehensiveness did not exceed CriticGPT working alone (the human's marginal contribution is fewer hallucinations, not more bugs caught); the comparison group was hourly-paid contractors, not senior engineers who know the codebase; and the paper's abstract volunteers that critics hallucinate bugs that "could mislead humans into making mistakes they might have otherwise avoided." An AI verifier is not a neutral filter — it injects new error signal into the process, which is precisely the term missing from Leike's "verifying a bug is easy" example: when the "bug" may itself be a hallucination, verifying that it is a real bug is not easy at all.
Footnote four: errors are converging — the independence premise is dissolving. Intuitively, even unreliable verifiers can backstop each other by voting — provided they are independent. ICML 2025's "Great Models Think Alike and this Undermines AI Oversight" reports two measurements: LLM-as-judge scores systematically favor models similar to the judge (self-preference generalized to affinity), and model mistakes are becoming more similar as capabilities increase, pointing to correlated-failure risk. In fairness: narrow self-preference showed weak and direction-inconsistent effects in a controlled software-engineering audit — it is not the dominant bias. The real threat is error correlation: when generator and verifier share blind spots, stacking verification layers does not reduce the joint error rate. That is the precise probabilistic meaning of "turtles all the way down." In-domain evidence points the same way: across 8 LLMs judging 2,686 code samples, "even the best-performing LLM frequently misjudges the correctness of the code"; another study finds systematic overcorrection — LLM reviewers frequently flagging correct code as defective, with more detailed prompts (requiring explanations and fixes) increasing misjudgment rates (structurally the CriticGPT nitpick problem: force the reviewer to say something and it will manufacture defects).
Even with unreliable AI verifiers, the turtles don't collapse if human oversight holds. The trouble is that "humans supervising automation" happens to be the field human-factors engineering has studied for forty years — with its most pessimistic conclusions.
Bainbridge's 1983 "Ironies of Automation" contains the "impossible task" passage that transfers to 2026 with only the nouns changed: "There is therefore no way in which the human operator can check in real-time that the computer is following its rules correctly. One can therefore only expect the operator to monitor the computer's decisions at some meta-level, to decide whether the computer's decisions are 'acceptable'. If the computer is being used to make the decisions because human judgement and intuitive reasoning are not adequate in this context, then which of the decisions is to be accepted? The human monitor has been given an impossible task." Substitute developer for operator and AI reviewer for automatic control system, and this is "humans reviewing the AI's review" as currently practiced. Parasuraman and Manzey (2010) add the harder sentence: automation complacency and automation bias occur in novices and experts, individuals and teams, and "cannot be prevented by training or instructions." "Strengthen human oversight," as a governance plan, collides head-on with forty years of human-factors evidence. Parasuraman & Riley's (1997) misuse/disuse framework predicted AI review's two failure modes exactly: too many false alarms and developers learn to ignore all AI comments (disuse — discarding the true positives too; Greptile's own CEO describing ~80% of critical bugs buried under "20 highly speculative guesses" is this mechanism verbatim), while a smooth experience turns approval into rubber-stamping (misuse).
Three 2026 findings annotate the script. The eye-tracking experiment (Wizard-of-Oz design): code labelled "LLM-generated" received more fixation time with no change in review thoroughness — the label changes attention's posture, not the quality of the gate. An interview study of 17 senior developers records a dangerous heuristic: using test results as guarantees of code correctness — outsourcing verification to yet another automation layer; one more turtle. And Cloudflare's production data (first-party, April 2026) offers a number that reads both ways: its self-built multi-agent review system ran 131,246 reviews in one month (48,095 MRs, $1.19 average per review), with engineers overriding the system's verdict on just 0.6% of MRs. Optimistic reading: the system is good enough to rarely need correction. Pessimistic reading: 0.6% is exactly what the rubber-stamp rate looks like in the misuse script. The same number cannot distinguish the two worlds — which is the proxy-metric disease itself. As for whether humans over-trust "AI-approved" code, no dedicated controlled experiment could be found as of this essay's cutoff; the gap is recorded as such.
Put the five sections together and the answer is not either/or — it is conditional.
Condition one: a correctness oracle independent of the LLM exists. Google's migration pattern is the template: deterministic AST/static analysis to locate targets, LLM to generate, automated unit-test verification loops, and everything through the same review pipeline as human code — the AI's output is clamped inside a non-LLM verification structure; the LLM is a pipeline stage, not the judge. The ~87% committed-unmodified figure, with review as the only bottleneck, happened inside that structure. Homogeneous changes with ready oracles (compilers, tests, type systems) — migrations, dependency upgrades, lint-grade fixes — are where AI review's evidence is most solid.
Condition two: false positives are measured and controlled as first-class. ByteDance added a ReviewFilter noise layer; the ICML 2025 deployment paper lists false-alarm reduction first among its four challenges and ships a built-in nitpick filter; GitHub moved from more-comments-is-better to "In 71% of the reviews, Copilot code review surfaces actionable feedback. In the remaining 29%, the agent says nothing at all" (self-reported). The production systems that survive all operate on the correct side of signal detection theory: control the false-alarm rate first, then talk about catch rate. Conversely, a tool that publishes only recall should be presumed to have precision that cannot survive publication.
Condition three: humans keep the verdict, and the cost structure makes sense. The Cloudflare pattern ($1.19 per review, full coverage, human override available) and Anthropic's counterfactual estimate (automated review would have caught roughly a third of the bugs behind past claude.ai incidents — vendor self-report) represent "AI review as a cheap additional layer, not a replacement layer." A dollar-per-run check with imperfect recall and near-zero marginal cost, stacked on top of human review, has almost inevitably positive expected value — provided it does not crowd out or anesthetize the human review underneath. That proviso is exactly what Meta's 5.5%-regression experiment and the automation-complacency literature demand be taken seriously.
Open-ended correctness judgment with no external oracle. When the question is "is this business logic right" rather than "is this API migration right," with no tests or type system underneath, LLM-verifying-LLM stands fully exposed to Section 5's four footnotes: unreliable verifiers, self-critique collapse, hallucination injection, converging errors. Stacking more AI verification layers there lowers perceived risk, not actual risk — and the gap between perceived and actual is the organizational version of the 40-point perception gap METR measured.
AI writes + AI reviews + humans exit the loop. The AIDev status quo (most AI PRs unreviewed by humans; agents dominating what review exists) plus Faros's +31.3% zero-review merges say this is not a hypothetical slippery slope but an equilibrium already drifting. The most radical public experiment is StrongDM's "Software Factory" charter — "Code must not be written by humans. Code must not be reviewed by humans." — replacing review with scenario verification held outside the codebase; upon publication, Hacker News readers promptly flagged suspected bugs and permissive error handling in its open-sourced code. A sample of one settles nothing, but it demonstrates the turtles thesis's core mechanism: when the verification layer shares its origin with the generation layer, system error rate is governed by their correlation — and that correlation is rising with model capability.
Activity metrics standing in for outcome metrics. "73.8% of comments resolved" and "PRs 42% slower" are simultaneously true; "7.5% of comments resolved by ML" and "nobody knows how many real bugs were caught" are simultaneously true; "0.6% override" and "cannot distinguish trust from rubber-stamping" are simultaneously true. Google's phrase — "easier-to-measure proxies" — belongs on the first page of every AI review procurement memo.
Supply-side behavior reveals practitioners' actual beliefs. Cursor paid a premium for Graphite — the company that writes code decided reviewing code is the next bottleneck, worth buying. GitHub moved Copilot code review to usage-based billing (June 2026) — usage grew past what the platform would subsidize, which is hard evidence of adoption. Google is sunsetting the consumer tier of Gemini Code Assist code review (effective July 17, 2026) — the first exit signal. And Greptile CEO Daksh Gupta, after raising $30M, published "There is an AI code review bubble," conceding that signal-to-noise is the industry's fundamental problem. Among the shovel-sellers, some have started telling the truth about the ore.
The essay reduces to eight claims, ordered by evidence strength:
What to watch by 2027: the direction and effect size of the first RCT on an AI review tool; whether Faros's zero-review merge share keeps climbing or gets governed back down; whether a Martian-style independent leaderboard can establish an evaluation protocol vendors cannot harvest; and whether any organization starts publishing full-chain data — AI comment → code change → change-failure rate. The previous essay said the verification bottleneck would not disappear, only move. This essay's finding is that it has moved onto the verifier. The next bottleneck is verifying the verifiers.
Big-tech first-party: Frömmgen et al., "Resolving Code Review Comments with Machine Learning" (ICSE-SEIP 2024) · Maddila et al., "AI-Assisted Fixes to Code Review Comments at Scale" (arXiv:2507.13499) · Nikolov et al., "How is Google using AI for internal code migrations?" (arXiv:2501.06972, ICSE-SEIP 2025) · Ziftci et al. (FSE 2025, arXiv:2504.09691) · BitsAI-CR (arXiv:2501.15134) · Mukherjee et al., CodeGuru (ECOOP 2022) · Cloudflare (blog.cloudflare.com/ai-code-review/, 2026-04) · GitHub, "60 million Copilot code reviews" (2026-03) · Anthropic, recursive-self-improvement (2026-05, vendor figures)
Independent academic: Chowdhury et al. (arXiv:2604.03196) · Sun et al., "Does AI Code Review Lead to Code Changes?" (arXiv:2508.18771) · Duma et al. (arXiv:2605.02273) · Khojah et al., eye tracking (arXiv:2606.26505) · Khan et al., benchmark survey (arXiv:2602.13377) · SWR-Bench (FSE 2026, arXiv:2509.01494) · Cihan et al., "Automated Code Review In Practice" (ICSE 2025 SEIP, arXiv:2412.18531) · Lu et al. (ICML 2025 Spotlight, arXiv:2505.17928) · Dhanorkar et al., oversight interviews (arXiv:2606.05391) · Monperrus, "The End of Code Review" (position, arXiv:2606.13175)
Verifier reliability: McAleese et al., CriticGPT (arXiv:2407.00215) · Ma et al., SAGA (arXiv:2507.06920) · Li et al., GV-consistency (ICLR 2024) · Stechly et al. (arXiv:2402.08115) · Goel et al., "Great Models Think Alike" (ICML 2025, arXiv:2502.04313) · Crupi et al. (arXiv:2507.16587) · Jin & Chen, overcorrection (arXiv:2603.00539) · Zhao et al., judge audit (arXiv:2604.16790) · Panickssery et al. (NeurIPS 2024) · Song et al., GV-Gap (ICLR 2025) · Leike et al. (arXiv:1811.07871)
Theory: Green & Swets (1966) / Stanislaw & Todorov (1999) · Parasuraman & Riley (Human Factors 1997) · Parasuraman & Manzey (Human Factors 2010) · Bainbridge, "Ironies of Automation" (Automatica 1983) · Simkute et al., "Ironies of Generative AI" (IJHCI 2024)
Benchmarks & vendors (all COI-flagged): greptile.com/benchmarks · Augment benchmark post · tenki.cloud/benchmarks/code-reviewer · entelligence.ai/code-review-benchmark-2026 · Qodo benchmark & Martian posts · Baz Martian post · CodeRabbit Martian post · deepsource.com/blog/ai-code-review-benchmarks · Martian Code Review Bench · pr-review-bench (dev.to; author employed at Sentry) · Sonar State of Code 2026 · Greptile, "There is an AI code review bubble" (2026-02) · StrongDM, "Software Factory" (2026-02) · Cursor/Graphite acquisition (2025-12)
Telemetry & surveys: Faros, "The Acceleration Whiplash" (2026-04, vendor telemetry) · Faros 2025 · LinearB 2026 Benchmarks · GitHub Octoverse 2025 · DORA 2024/2025 · JetBrains Developer Ecosystem 2025 · Stack Overflow 2025 · METR (arXiv:2507.09089 + 2026-02 update)