Research map / Large language models / AI and society
The Leaderboard Illusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Audits Chatbot Arena (LMArena) over Jan 2024–Apr 2025 and shows undisclosed private multi-variant testing with score retraction, unequal data access, and silent model deprecation all distort Arena rankings, with simulations and a real submission experiment supporting the claims.
Problem
Chatbot Arena has become the de facto leaderboard for LLMs, but its rankings may be distorted by undisclosed private testing with selective score retraction, asymmetric data access across providers, and opaque model deprecation that violate Bradley-Terry assumptions.
Approach
Combines crawled battles, leaderboard snapshots, and provider-shared battle data with simulations of best-of-N selection, Bradley-Terry sensitivity analyses, and real submissions of private Aya-Vision variants to quantify how selective disclosure, sampling asymmetry, and deprecation inflate or destabilize Arena scores; also fine-tunes a 7B model with varying Arena data to estimate overfitting gain
Key results
- Meta, Google, OpenAI, and others were found to privately test many variants; Meta tested 27 private variants on the main Arena in Jan-Mar 2025 (43 including vision).
- Training on 70% Arena data raised ArenaHard win-rate vs Llama-3.1-8B-Instruct from 23.5% to 49.9%, while MMLU did not improve (66.5% to 65.9%).
- Of 243 models, 205 were silently deprecated versus 47 officially listed; 87.8% of open-weight and 89% of open-source models were deprecated vs 80% of proprietary.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase Arena data in the training mixture from 0% to 70% (ablation) | 7B base model (Cohere Command family) · ArenaHard | win-rate vs. Llama-3.1-8B-Instruct: 23.5% -> 49.9% | Helped |
| Increase Arena data in the training mixture from 0% to 30% (ablation) | 7B base model (Cohere Command family) · ArenaHard | win-rate vs. Llama-3.1-8B-Instruct: 23.5% -> 42.7% | Helped |
| Increase Arena data in the training mixture from 0% to 30% (ablation) | 7B base model (Cohere Command family) · MMLU | score: 66.5% -> 64.4% | Hurt |
| Increase Arena data in the training mixture from 0% to 70% (ablation) | 7B base model (Cohere Command family) · MMLU | score: 66.5% -> 65.9% | Mixed |
| Select the higher-scoring submission from two identical checkpoints instead of reporting one submission | Aya-Vision-8B · Chatbot Arena | Arena Score: 1052 (±21/22) -> 1069 (±19/23) | Helped |
| Select the higher-scoring variant from two checkpoints | Aya-Vision-32B · Chatbot Arena | Arena Score: 1059 (±18/23) -> 1097 (±29/25) | Helped |
| Test more private variants and select the best-performing score | Chatbot Arena model submissions · Chatbot Arena | maximum Arena Score: approximately 100 points | Helped |
| Increase private variants from 0 to 20 | Simulated model submissions · Chatbot Arena | maximum Arena Score: approximately 50 points | Helped |
| Increase the number of tested variants from 1 to 3 | Cohere model variants · Chatbot Arena | share of prompts collected: 5.9% -> 19.4% | Helped |
| Deprecate Model D halfway through the battle history under a changing task distribution | Simulated Bradley-Terry rankings · Two-phase task distribution simulation | ranking reliability | Hurt |
| Use a sparse or disconnected comparison graph instead of a dense connected graph | Simulated Bradley-Terry rankings · Sparse battle history simulation | ranking accuracy | Hurt |
Surprising findings
- Fine-tuning on Arena data improved ArenaHard win-rate but MMLU slightly declined (66.5% to 64.4% at 30% arena data), suggesting gains are narrow.
- Two identical Aya-Vision-8B checkpoints received different Arena scores (1069 vs 1052), showing selection noise alone can yield advantage.
Limitations
- Private variant tracking covers only Jan-Mar 2025 scrape and may undercount; some private models could not be de-anonymized.
- Overfitting experiment is a lower-bound illustration with a single 7B model, no hyperparameter tuning, and LLM-as-judge evaluation.
- Several authors submitted models to the Arena, a potential conflict of interest.
Experimental setup
Datasets: arena-human-preference-100k historical-battles (1.8M battles, Apr 2023-Jan 2025) scraped-random-sample (5.8K battles) API prompts (197K single-turn, Cohere) leaderboard-stats (14.3K records) proprietary other-sft-mix
Benchmarks: ArenaHard MMLU
Metrics: win-rate Arena score (Bradley-Terry) sampling rate deprecation rate prompt duplication rate
Baselines: Llama-3.1-8B-Instruct
Builds on: Bradley-Terry model Elo rating ArenaHard FastChat
Closest papers by meaning
- Improving Your Model Ranking on Chatbot Arena by Vote Rigging (ICML 2025 · similarity 0.88)
- Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards (ICML 2025 · similarity 0.87)
- Position: AI Competitions Provide the Gold Standard for Empirical Rigor in GenAI Evaluation (ICML 2025 · similarity 0.86)
- Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation (ICLR 2026 · similarity 0.85)
- SWE-ABS: Adversarial Benchmark Strengthening Exposes Inflated Success Rates on Test-based Benchmark (ICML 2026 · similarity 0.85)
- A Unified Perturbation Framework for Analyzing Leaderboard Stability and Manipulation (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.