Research map / Large language models / Reasoning and chain of thought
Active Evaluation Acquisition for Efficient LLM Benchmarking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Active Evaluation Acquisition (AEA), which uses a neural process to model dependencies across benchmark prompts and an RL policy (PPO with intermediate rewards) to pick a small prompt subset per model, predicting the remaining scores.
Problem
Comprehensive LLM benchmarks with hundreds to thousands of prompts are costly to run; the goal is to estimate a model's benchmark score accurately while evaluating on far fewer prompts, including in cold-start settings where new prompts lack any scores.
Approach
Model prompt-score dependencies with an attentive/Set-Transformer neural process, then learn a sequential RL acquisition policy (PPO) that chooses which prompts to evaluate per model, using intermediate rewards from prediction-error improvement and neural-process auxiliary information; the policy scores candidate prompts via inner products with their embeddings so it generalizes to new prompts.
Key results
- RL policy achieves lowest estimation error on all five benchmarks, e.g., AlpacaEval 0.001±0.000 at K=100 vs 0.005 for uniform and 0.003 for C-IRT.
- On MMLU (K=100) the RL policy with neural-process prediction reaches 0.013±0.000 error vs 0.018 for uniform sampling and 0.014 for C-Score.
- Versus TinyBenchmarks on Open LLM (K=200), RL with NP gets 0.018±0.001 vs 0.022±0.004 for TB with NP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RL-based active prompt acquisition instead of the strongest reported baseline | AEA · AlpacaEval 2.0 (K=100) | absolute error: 0.003\pm0.001 -> 0.001\pm0.000 | Helped |
| use RL-based active prompt acquisition instead of the strongest reported baseline | AEA · HELM-Lite (K=200) | absolute error: 0.035\pm0.012 -> 0.030\pm0.005 | Helped |
| use RL-based active prompt acquisition instead of the strongest reported baseline | AEA · Open LLM (K=200) | absolute error: 0.022\pm0.002 -> 0.018\pm0.001 | Helped |
| use RL-based active prompt acquisition instead of the strongest reported baseline | AEA · MMLU (K=100) | absolute error: 0.014\pm0.002 -> 0.013\pm0.000 | Helped |
| use RL-based active prompt acquisition instead of the strongest reported baseline | AEA · Chatbot Arena (K=40) | absolute error: 0.035\pm0.003 -> 0.034\pm0.006 | Helped |
| predict the missing evaluation scores rather than aggregate acquired scores only (ablation) | RL acquisition · AlpacaEval 2.0 | absolute error: 0.064\pm0.006 -> 0.001\pm0.000 | Helped |
| predict the missing evaluation scores rather than aggregate acquired scores only (ablation) | RL acquisition · HELM-Lite | absolute error: 0.081\pm0.019 -> 0.030\pm0.005 | Helped |
| predict the missing evaluation scores rather than aggregate acquired scores only (ablation) | RL acquisition · Open LLM | absolute error: 0.063\pm0.018 -> 0.018\pm0.001 | Helped |
| predict the missing evaluation scores rather than aggregate acquired scores only (ablation) | RL acquisition · MMLU | absolute error: 0.050\pm0.006 -> 0.013\pm0.000 | Helped |
| predict the missing evaluation scores rather than aggregate acquired scores only (ablation) | RL acquisition · Chatbot Arena | absolute error: 0.045\pm0.007 -> 0.034\pm0.006 | Helped |
| add neural-process auxiliary information to PPO (ablation) | PPO acquisition · AlpacaEval | absolute error: 0.004\pm0.002 -> 0.003\pm0.001 | Helped |
| add intermediate rewards to PPO acquisition (ablation) | PPO acquisition · MMLU | absolute error: 0.017\pm0.004 -> 0.013\pm0.000 | Helped |
Surprising findings
- Uncertainty sampling and information gain policies consistently fail to explore the action space and perform worse than random sampling.
- Combinatorial optimization static policy performs poorly even on small benchmarks, attributed to train-test model distribution shift.
Limitations
- Performance degrades under model bias (train/test from different organizations), requiring more acquisitions.
- Cold-start estimates remain less accurate than in fully observed setting; semi-supervised pseudo-labeling is only one tried approach.
- Prompt embedding quality affects results; no thorough hyperparameter search was conducted.
Experimental setup
Datasets: HuggingFace Open LLM Leaderboard MMLU HELM-Lite AlpacaEval 2.0 Chatbot Arena MT-Bench
Benchmarks: HuggingFace Open LLM Leaderboard MMLU HELM-Lite AlpacaEval 2.0 Chatbot Arena
Metrics: absolute error in benchmark score estimation win rate
Baselines: Uniform sampling Stratified random sampling Clustering-Embed Clustering-Score Clustering-IRT Combinatorial optimization Uncertainty sampling Information gain TinyBenchmarks
Builds on: Attentive Neural Process Set Transformer PPO tinybenchmarks active learning
Closest papers by meaning
- tinyBenchmarks: evaluating LLMs with fewer examples (ICML 2024 · similarity 0.92)
- RocketEval: Efficient automated LLM evaluation via grading checklist (ICLR 2025 · similarity 0.91)
- How Benchmark Prediction from Fewer Data Misses the Mark (NeurIPS 2025 · similarity 0.91)
- On Speeding Up Language Model Evaluation (ICLR 2025 · similarity 0.91)
- Scaling Up Active Testing to Large Language Models (NeurIPS 2025 · similarity 0.90)
- Efficient Evaluation of LLM Performance with Statistical Guarantees (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.