Research map / Large language models / Agents, code and math
Unbiased Evaluation of Large Language Models from a Causal Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that Agents-as-an-Evaluator (LLM question rephrasing) carries data and model biases, and proposes Unbiased Evaluator, which evaluates LLMs via bags of atomic interventions (BOAT) on MCQ and math benchmarks to reduce contamination and bias.
Problem
Benchmark contamination undermines LLM evaluation, and agent-based (Agents-as-an-Evaluator) methods that generate new questions may themselves introduce unexplored evaluation biases.
Approach
Formulates evaluation bias as decomposable into original, related, and independent terms, and replaces question rewriting with rule-based atomic interventions (e.g., distractor hints, option shuffling, label replacement, binary transformation) combined into bags that are applied to existing benchmarks to produce a causal, interpretable evaluation.
Key results
- Unbiased Evaluator lowers accuracy for all models versus vanilla, e.g., MMLU drop of 14.69 points for GPT-4o and 22.12 points on ARC-C for Yi1.5-34B-Chat.
- Human verification of intervened questions reports 99.3% (ARC-C), 99.9% (MMLU), 99.7% (GSM8K) accuracy.
- Unbiased Evaluator rankings correlate with LiveBench at Kendall 1.000 (vs 0.600 for vanilla) on the 8 models covered by LiveBench.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate GPT-4-Turbo with Unbiased Evaluator instead of the vanilla protocol | GPT-4-Turbo · ARC-C | accuracy: 96.48 -> 89.74 ± 0.13 | Hurt |
| Evaluate Gemini-2.0 with Unbiased Evaluator instead of the vanilla protocol | Gemini-2.0 · MMLU | accuracy: 86.20 -> 73.57 ± 0.09 | Hurt |
| Evaluate Gemini-2.0 with Unbiased Evaluator instead of the vanilla protocol | Gemini-2.0 · GSM8K | accuracy: 97.88 -> 89.35 ± 0.04 | Hurt |
| Fine-tune on the original MMLU test set and evaluate with Unbiased Evaluator rather than the vanilla protocol (ablation) | Llama2-13B · MMLU test | accuracy: 96.6 -> 37.1 | Hurt |
| Use Unbiased Evaluator scores instead of vanilla scores | Unbiased Evaluator · LiveBench global average scores | Kendall correlation: 0.600 -> 1.000 | Helped |
| Use Unbiased Evaluator scores instead of vanilla scores | Unbiased Evaluator · LiveBench global average scores | Pearson correlation: 0.918 -> 0.949 | Helped |
| Use Unbiased Evaluator rather than Agents-as-an-Evaluator (ablation) | Four evaluated LLMs · MMLU subject domains | Pearson correlation between domain accuracy and consensus-error rate: -0.47 -> 0.07; -0.38 -> 0.16; -0.45 -> 0.15; -0.41 -> 0.10 | Helped |
| Increase rephrasing strength in Agents-as-an-Evaluator | Agents-as-an-Evaluator · ARC-C and MMLU probing tasks | over-confidence rate | Hurt |
| Increase intervention strength in Unbiased Evaluator (ablation) | Unbiased Evaluator · ARC-C and MMLU probing tasks | over-confidence rate | No clear effect |
| Apply Binary Transformation (ablation) | Qwen2.5-7B-Instruct · ARC-C | accuracy | Hurt |
| Apply Option Shuffling (ablation) | Qwen2.5-7B-Instruct · ARC-C | accuracy | Hurt |
| Apply Label Replacement (ablation) | Qwen2.5-7B-Instruct · ARC-C | accuracy | Hurt |
Surprising findings
- Fine-tuning Llama2-13B on the original MMLU test set raises accuracy to 96.6%, but under the Unbiased Evaluator it reaches only 37.1%.
Limitations
- Interventions are rule-based and mainly applicable to tasks with natural structure (MCQ, math); open-ended tasks are not covered.
- Evaluation reliability is validated against LiveBench on only a small set of models (excluding GPT-4-Turbo and Yi1.5-34B-Chat).
- Bias analysis relies on LLM self-confidence scores, and the theoretical proposition is not validated empirically beyond the probing experiments.
Experimental setup
Datasets: ARC-Challenge MMLU GSM8K
Benchmarks: ARC-Challenge MMLU GSM8K
Metrics: accuracy Pearson correlation Kendall correlation consensus-error rate over-confidence rate under-confidence rate
Baselines: Agents-as-an-Evaluator Vanilla benchmark evaluation LiveBench
Builds on: Agents-as-an-Evaluator LLM-as-a-Judge OpenCompass
Closest papers by meaning
- An Empirical Analysis of Uncertainty in Large Language Model Evaluations (ICLR 2025 · similarity 0.90)
- ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate (ICLR 2024 · similarity 0.89)
- Any Large Language Model Can Be a Reliable Judge: Debiasing with a Reasoning-based Bias Detector (NeurIPS 2025 · similarity 0.89)
- Accelerating Unbiased LLM Evaluation via Synthetic Feedback (ICML 2025 · similarity 0.89)
- AgentBench: Evaluating LLMs as Agents (ICLR 2024 · similarity 0.88)
- RocketEval: Efficient automated LLM evaluation via grading checklist (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.