Research map / Large language models / AI and society
Fantastic Bugs and Where to Find Them in AI Benchmarks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a measurement-theoretic framework (tetrachoric correlation, item scalability, item-total correlation) to flag potentially invalid benchmark questions from LLM response patterns for expert review, plus an LLM-judge first pass, applied to nine benchmarks.
Problem
Benchmarks contain invalid questions (ambiguous wording, incorrect answer keys, grading issues) that distort model rankings, but manually reviewing thousands of items is infeasible.
Approach
Assuming the sum/mean score is a sufficient statistic implies a unidimensional Rasch-type structure, which yields expected non-negative inter-item and item-total correlations; items whose statistics fall outside these expected ranges are flagged for expert review, and an LLM judge provides a first-pass review.
Key results
- Measurement-theoretic signals outperform variance and Fleiss' Kappa baselines in sensitivity on GSM8K (88 invalid of 997 by GSM8K-Platinum labels).
- Expert review confirms up to 84% of top-50 flagged questions exhibit substantive flaws across nine benchmarks.
- LLM-judge (ChatGPT O1) reached 98% precision in identifying invalid questions among the first 100 GSM8K questions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use measurement-theoretic signals to flag questions | AI benchmark revision · GSM8K | Sensitivity | Helped |
| Use tetrachoric correlation to rank questions for expert inspection | Benchmark revision · Nine benchmarks | Precision@50: up to 84\% | Helped |
| Use an LLM judge for first-pass review | GSM8K benchmark audit · GSM8K, first 100 questions | Precision: 98\% | Helped |
| Increase the number of LLM responses | Measurement-theoretic detection · GSM8K | Precision@50 | Helped |
| Increase diversity of LLM creator organizations, model sizes, and release dates | Measurement-theoretic detection · GSM8K | Precision@50 | Helped |
| Use AND Vote instead of OR Vote to combine binary anomaly flags | Measurement-theoretic signals · GSM8K | Sensitivity | Mixed |
Surprising findings
- Grading issues (e.g., answer format mismatches) account for most invalid GSM8K items in the LLM-judge check (93.3% of invalid questions in the 100-question sample).
- No single statistical method detects all invalid questions; each flags different sets.
Limitations
- Statistical anomalies may not align with human judgments of invalidity, e.g., cultural ambiguity.
- Validity criteria chosen determine which questions are flagged; content and consequential validity are unaddressed.
- Limited number and homogeneity of LLM responses reduce detection power.
Experimental setup
Datasets: GSM8K GSM8K-Platinum MMLU MMLU HS-Math AIR-Bench ThaiExam MedQA MMLU Cli-Know MMLU Pro-Med OpenbookQA MMLU 5-Sub HELM leaderboard responses
Benchmarks: GSM8K MMLU HS-Math AIR-Bench ThaiExam MedQA MMLU Cli-Know MMLU Pro-Med OpenbookQA MMLU 5-Sub
Metrics: Sensitivity Precision@k
Baselines: variance in predictions Fleiss' Kappa Gaussian Rank Mean ensemble OR Vote AND Vote Majority Vote
Builds on: Rasch model classical test theory Mokken scaling item response theory
Closest papers by meaning
- BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices (NeurIPS 2024 · similarity 0.90)
- Establishing Best Practices in Building Rigorous Agentic Benchmarks (NeurIPS 2025 · similarity 0.87)
- Reliable and Efficient Amortized Model-based Evaluation (ICML 2025 · similarity 0.86)
- AI Evaluation Should Require Standardized Item-Level Data Releases (NeurIPS 2026 · similarity 0.86)
- AutoBencher: Towards Declarative Benchmark Construction (ICLR 2025 · similarity 0.85)
- MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.