Research map / Large language models / Alignment and preferences
From Crowdsourced Data to High-quality Benchmarks: Arena-Hard and Benchbuilder Pipeline
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces BenchBuilder, an LLM-based pipeline that curates challenging open-ended prompts from crowdsourced data, releases the 500-prompt Arena-Hard benchmark, and proposes new separability and agreement metrics for benchmark quality.
Problem
Static and expert-curated benchmarks are expensive to build and saturate or leak, while crowdsourced live evaluations like Chatbot Arena are costly and slow for frequent model evaluation; open-ended benchmarks also need quality-controlled prompts and metrics that capture separability and human align
Approach
Score each crowdsourced prompt with an LLM annotator on seven quality criteria (specificity, domain knowledge, complexity, problem-solving, creativity, technical accuracy, real-world application), cluster prompts by topic with BERTopic, keep high-scoring clusters, and sample evenly to build a benchmark judged by an LLM judge against a baseline model; also proposes Separability with Confidence, Agr
Key results
- Arena-Hard achieves 87.4% separability and 90.9% confidence agreement with Chatbot Arena rankings at about $20 per model evaluation, versus 22.6% separability for MT-Bench.
- With style control, Arena-Hard reaches 98.6% confidence agreement and Spearman correlation with style-controlled Chatbot Arena English Hard Prompt rankings.
- An ensemble of GPT-4-Turbo and Gemini-1.5-Pro judges reaches 89.5% separability and 91.5% confidence agreement, and the WildChat-derived set reaches 88.6% agreement vs 36.4% for random prompts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Curate challenging prompts from Chatbot Arena with BenchBuilder instead of using MT-Bench | Benchmark evaluation of top-20 Chatbot Arena models · Top-20 Chatbot Arena models | Separability: 22.6% -> 87.4% | Helped |
| Curate challenging prompts from Chatbot Arena with BenchBuilder instead of randomly sampling WildChat prompts | WildChat benchmark evaluation · WildChat models | Confidence Agreement: 36.4% -> 88.6% | Helped |
| Add style control to model scoring (ablation) | Arena-Hard · Style-controlled Chatbot Arena Hard Prompt models | Spearman Correlation: 94.9% -> 98.6% | Helped |
| Add style control to model scoring (ablation) | Arena-Hard · Style-controlled Chatbot Arena Hard Prompt models | Confidence Agreement: 94.4% -> 98.6% | Helped |
| Use GPT-4-Turbo and Gemini-1.5-Pro as an ensemble judge rather than GPT-4-Turbo alone (ablation) | Arena-Hard · Arena-Hard models | Spearman Correlation: 93.2% -> 96.5% | Helped |
| Replace GPT-4-Turbo with Claude-3-Opus as judge (ablation) | Arena-Hard · Arena-Hard models | Confidence Agreement: 90.9% -> 66.7% | Hurt |
| Replace GPT-4-Turbo with Gemini1.5-Pro as judge (ablation) | Arena-Hard · Arena-Hard models | Brier Score: 0.069 -> 0.064 | Helped |
| Replace GPT-4-Turbo with Llama3-70B as judge (ablation) | Arena-Hard · Arena-Hard models | Confidence Agreement: 90.9% -> 65.6% | Hurt |
| Use Llama-3-70B-Instruct instead of GPT-4-Turbo to annotate prompt quality (ablation) | BenchBuilder · LlamaSet versus random prompt baselines | Separability: 68.9% -> 84.4% | Helped |
| Prompt Llama-3.1-70B-Instruct to provide more detail (ablation) | Llama-3.1-70B-Instruct evaluated by GPT-4-Turbo without style control · Arena-Hard | Score: 44.5 -> 53.5 | Helped |
| Prompt Llama-3.1-70B-Instruct to provide more detail with style control enabled (ablation) | Llama-3.1-70B-Instruct evaluated by GPT-4-Turbo · Arena-Hard | Score: 41.7 -> 39.8 | Hurt |
| Apply style control to judge scores (ablation) | Arena-Hard verbosity evaluation · Model score versus average token length | Pearson correlation: 0.364 -> 0.193 | Helped |
Surprising findings
- MT-Bench has high Spearman correlation (89.9%) with Chatbot Arena yet only 22.6% separability and 26.6% confidence agreement, showing rank correlation hides ranking granularity.
- Prompting models to be more verbose or use more markdown raises scores on the uncontrolled benchmark, but style control neutralizes this gain.
Limitations
- Curated prompts skew toward technical domains; the seven qualities may not capture all attributes.
- Evaluation covers only single-turn English prompts; multi-turn and non-English are not evaluated.
- Self-bias toward GPT models remains a concern when GPT-4-Turbo is the sole judge.
Experimental setup
Datasets: Chatbot Arena WildChat-1M
Benchmarks: Arena-Hard MT-Bench AlpacaEval 2.0 LC Chatbot Arena English Hard Prompts
Metrics: Separability with Confidence Agreement with Confidence Pair Rank Brier Score Spearman correlation Kendall Tau correlation
Baselines: MT-Bench AlpacaEval 2.0 LC Chatbot Arena Random sample baselines Wild-Random-250
Builds on: Chatbot Arena LLM-as-a-Judge BERTopic Bradley-Terry model MT-Bench
Closest papers by meaning
- WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild (ICLR 2025 · similarity 0.90)
- MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures (NeurIPS 2024 · similarity 0.90)
- BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks (NeurIPS 2025 · similarity 0.90)
- Automating Dataset Updates Towards Reliable and Timely Evaluation of Large Language Models (NeurIPS 2024 · similarity 0.89)
- JudgeLM: Fine-tuned Large Language Models are Scalable Judges (ICLR 2025 · similarity 0.89)
- AutoBencher: Towards Declarative Benchmark Construction (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.