Research map / Large language models / Language, safety and interpretability

Limits to scalable evaluation at the frontier: LLM as judge won’t beat twice the data

ICLR 2025 · Large language models › Language, safety and interpretability · approach: Uncertainty quantification for LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows theoretically that with a judge no more accurate than the evaluated model, debiasing LLM-judge scores (PPI-style) can save at most about half the ground-truth labels (sample efficiency factor ≤ 2), and confirms this empirically on MMLU, MT-Bench and TruthfulQA.

Problem

Whether LLM-as-a-judge combined with debiasing methods (e.g., prediction-powered inference) can substantially reduce the number of costly ground-truth labels needed to evaluate newly released frontier models that may be better than the judge.

Approach

Proves that the optimal sample efficiency factor of any unbiased estimator using judge scores is bounded by 1/(1-rho^2), where rho is the Pearson correlation between true and proxy scores, and shows that when the judge's agreement is at most the evaluated model's accuracy, rho^2 ≤ 0.5, so gains are at most a factor of two.

Key results

• Theorem: when 0.5 <= agreement <= evaluated model accuracy, the squared correlation rho^2 is at most 0.5, so any unbiased estimator's sample efficiency factor is at most 2.
• On MMLU, sample efficiency factor tau_max stays below 2 except when GPT-4-class judges evaluate much weaker LLaMa2-7b (where agreement greatly exceeds accuracy).
• On MT-Bench with GPT-4 as judge, tau_max gets close to 2 in some cases but consistently stays below it.

Measured effects reported in the paper

Surprising findings

• High agreement rate with the ground truth does not guarantee sample efficiency gains; there are settings with zero judge bias where tau_max equals 1.
• Model rankings on MMLU are preserved near-perfectly with a 60%-accurate judge making random errors, despite low agreement.

Limitations

• Results apply to uniform (non-adaptive) sampling and binary proxy scores; the non-binary extension gives only a weaker bound. Gains above factor two are possible when judging is substantially easier than the task itself. The agreement-versus-accuracy indicator assumes judge and evaluated model face

Experimental setup

Datasets: MMLU MT-Bench TruthfulQA HELM leaderboard predictions

Benchmarks: MMLU MT-Bench TruthfulQA

Metrics: sample efficiency factor tau_max squared Pearson correlation rho^2 model ranking correctness

Baselines: classical ground-truth estimator PPI stratified PPI

Builds on: prediction-powered inference control variates

Closest papers by meaning

• Any Large Language Model Can Be a Reliable Judge: Debiasing with a Reasoning-based Bias Detector (NeurIPS 2025 · similarity 0.89)
• How Benchmark Prediction from Fewer Data Misses the Mark (NeurIPS 2025 · similarity 0.87)
• Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge (ICLR 2025 · similarity 0.87)
• Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators (ICML 2025 · similarity 0.87)
• SEBRA : Debiasing through Self-Guided Bias Ranking (ICLR 2025 · similarity 0.87)
• RocketEval: Efficient automated LLM evaluation via grading checklist (ICLR 2025 · similarity 0.86)