Using an LLM to score another LLM's outputs (LLM-as-Judge) is already the default infrastructure for evaluation, reward models, RLHF, and agent self-critique. But the judge is not an impartial ruler—it is systematically pulled off by surface artifacts, and it favors itself. Today we make it clear: where these biases come from, why "let the model judge right vs. wrong" still holds in principle, and how to calibrate the judge back at the mechanism level.
Picture a code reviewer who is supposed to judge only quality, but unconsciously rubber-stamps the PR that's listed first, or the one with more lines. That's not laziness—his judgment is contaminated by surface features unrelated to quality. Closer to your world: a query planner that shouldn't depend on join order, yet switches its execution plan just because you wrote table A first—the result depends on input arrangement it should ignore. LLM judges behave exactly like this.
You want the judge to answer "is A or B the better answer." An ideal judge looks only at quality. But Zheng et al. 2023 (the MT-Bench paper) systematically measured three biases:
The root is next-token prediction: models absorb many spurious correlations from training—"earlier option" and "longer text" happened to often coincide with "the chosen one" in human-labeled data, so the model mistook correlation for causation. Those signals leak into its "quality judgment."
This is why a single naive LLM score is untrustworthy: you may be measuring "who's listed first," not "who's better." The first mechanism-level fix is order symmetrization—run both orders, and only count the verdict when the two agree.
from anthropic import Anthropic client = Anthropic() # needs ANTHROPIC_API_KEY def judge(question, first, second): # return only the winner label A / B prompt = f"Q:{question}\n\n[Answer A]{first}\n\n[Answer B]{second}\n\nReply only A or B:" r = client.messages.create(model="claude-opus-4-8", max_tokens=2, messages=[{"role":"user","content":prompt}]) return r.content[0].text.strip() # Order symmetrization: ask both orders, detect position bias v1 = judge(q, ans_x, ans_y) # X first v2 = judge(q, ans_y, ans_x) # Y first (label mirrors) winner_1 = "X" if v1=="A" else "Y" winner_2 = "Y" if v2=="A" else "X" if winner_1 == winner_2: print("Robust winner:", winner_1) else: print("Position bias! Call it a tie or send to human review") # disagree = don't trust
Letting the same person who wrote the code write its unit tests—they'll test exactly the cases they already thought of and miss their blind spots, because the bug and the tests share one mental model. More precisely, in distributed systems: if a primary and its replica run the same buggy firmware, their failures are correlated—the replica can't independently validate the primary, because both fail at the same place. When the "model that generated the answer" and the "model acting as judge" are the same (or same-family), that is this same failure domain.
Zheng et al. observed GPT-4 as judge gives its own outputs about a 10% higher win rate. This isn't mere "vanity." Panickssery et al. 2024 (LLM Evaluators Recognize and Favor Their Own Generations) gives the mechanism:
This leads directly to correlated errors: if judge and evaluatee share architecture/training data, their blind spots overlap—the judge is exactly blind to the kind of errors the generator makes, because it would make them too. The "pass" it issues is false safety: not "the answer is right," but "two same-model instances erred at the same place together." Using one model for "generate + self-judge" is essentially letting a firmware bug sign its own health certificate.
# Measure self-preference: same answers, compare "self" vs "other" judging def win_rate_for(judge_model, my_ans, other_ans, q): wins = 0 for a, b, question in zip(my_ans, other_ans, q): v = judge_with(judge_model, question, a, b) # order-symmetrized wins += (v == "mine") return wins / len(q) # Judge with itself vs. a different-family model as judge self_rate = win_rate_for("model-X", x_ans, y_ans, q) # X judges X's answers neutral = win_rate_for("model-Z", x_ans, y_ans, q) # neutral third party print(f"self win {self_rate:.2f} vs neutral {neutral:.2f}") # if self_rate >> neutral → self-preference exists, don't trust self-judging
Given all these biases and imperfect generation, why trust the model as a judge at all? The answer hides in an intuition you know: verifying a solution is usually easier than finding it—the everyday version of P vs NP. Checking a Sudoku is instant; solving it is hard. Same in backend: writing an assert / validator is far cheaper than writing the correct implementation; a linter catches errors in correct code it could never write. "Verification is easier than generation" is the bedrock that makes LLM-as-Judge possible.
Formalized: the Generation-Verification Gap (GV-Gap) = how much performance improves when you re-weight a batch of the model's generations by its own verification scores. Intuition: sampling one answer directly has accuracy p_gen; but generating N and using "verification ability" to pick the best raises it to p_ver. GV-Gap = p_ver − p_gen, measuring how much the model's "reviewing" precision exceeds its "writing" precision.
This gap is the engine behind Best-of-N, reward models, self-refine, and o1-style reasoning—generate a lot, then filter/rank with a verification signal. Song et al. 2024 (Mind the Gap: Examining the Self-Improvement Capabilities of LLMs, arXiv 2412.02674) proposes GV-Gap as a core metric of self-improvement, and finds a variant of it scales monotonically with pretraining compute—bigger models "review" relatively better than they "write."
But the gap is not universal. For hard-enough reasoning, the model can't verify its own errors—judging its own blind spots, GV-Gap collapses to 0 or negative (this is why self-refine often "makes things worse"). When GV-Gap > 0 is exactly the boundary of "when LLM-as-Judge is trustworthy": big gap → the judge is useful; collapsed gap → it degrades into the correlated errors from the previous card.
# Estimate GV-Gap: accuracy of generation vs. "self-verify pick best" def gv_gap(model, tasks, N=8): p_gen, p_ver = 0, 0 for t in tasks: cands = [generate(model, t.q) for _ in range(N)] # generation baseline: correctness of just the first p_gen += t.is_correct(cands[0]) # verify-pick-best: model scores each candidate, take the top scores = [verify_score(model, t.q, c) for c in cands] best = cands[scores.index(max(scores))] p_ver += t.is_correct(best) n = len(tasks) return p_ver/n - p_gen/n # >0 means "review" beats "write" → judging is meaningful
Don't trust a single node. The old distributed wisdom: use cross-failure-domain replica voting (quorum) to resist correlated failures—N-version programming has different teams independently implement one spec, then vote, because independent implementations won't fail at the same place. Treat "different-family LLMs" as replicas from different failure domains, have each score and then aggregate—that's canceling the prior three cards' biases with diversity. Also calibrate the "gauge": if the judge's "9" doesn't correspond to a real 90% correctness, that number is a broken scale.
A single judge carries all of position bias, verbosity bias, self-preference, and correlated errors. Two mechanism-level fixes:
Note diversity is the prerequisite: if the panel is all same-family models (or one model run thrice), correlated errors aren't solved—three replicas fail together, and voting just casts the same bias three times. You want decorrelation, not count.
# Diverse jury: different-family models each score, aggregate by vote JURY = ["claude-opus-4-8", "gpt-judge", "gemini-judge"] # key: different families def panel_verdict(q, ans_a, ans_b): votes = [] for m in JURY: # each judge order-symmetrizes to remove position bias v1 = judge_with(m, q, ans_a, ans_b) v2 = judge_with(m, q, ans_b, ans_a) if v1 == v2: # count only if both orders agree votes.append(v1) if not votes: return "tie" # all position-unstable → call it a tie # majority vote: different-domain biases cancel here return max(set(votes), key=votes.count)