Research map / Large language models / Language, safety and interpretability
Benchmarking LLMs via Uncertainty Quantification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Benchmarks open-source LLMs by pairing accuracy with conformal-prediction set size as an uncertainty metric across five MCQA-formatted NLP tasks.
Problem
Open LLM leaderboards rank models by accuracy alone and ignore uncertainty, so models with equal accuracy can differ in how certain they are.
Approach
Applies split conformal prediction (LAC and APS scores) to LLM option-letter softmax probabilities to get distribution-free, coverage-guaranteed prediction sets; set size serves as a rigorous uncertainty measure reported alongside accuracy.
Key results
- Qwen-14B achieves the highest average accuracy (74.00%) and lowest average set size (2.17) among 6B-14B models, while Falcon-7B has the lowest accuracy (24.84%) and largest set size (3.75).
- Conformal prediction achieves average coverage above 90% for all evaluated models (e.g., 92.94% for Qwen-14B), and yields lower average ECE (8.35%) than entropy (8.46%) and maximal probability (8.61%) on InternLM-7B.
- Instruction-tuned Llama-2 chat variants show higher uncertainty than base models, and some higher-accuracy models (e.g., InternLM-7B vs MPT-7B on DRS, 19.34 points) exhibit larger set sizes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a larger Qwen model rather than Qwen-1.8B (scale to Qwen-14B) | Qwen series · Most tasks | accuracy | Helped |
| Scale Qwen from 1.8B to 14B | Qwen series · Most tasks | set size | Helped |
| Scale Qwen from 14B to 72B | Qwen series · RC and DRS | set size | Hurt |
| Use instruction-finetuned Chat-V1 formatting instead of the base model | Llama-2 series · Five-task average | accuracy | Hurt |
| Use instruction-finetuned Chat-V1 formatting instead of the base model | Llama-2 series · Five-task average | set size | Hurt |
| Use instruction-finetuned Chat-V2 formatting instead of the base model | Llama-2-7B and Llama-2-13B · Five-task average | accuracy | Helped |
| Use instruction-finetuned Chat-V2 formatting instead of the base model | Llama-2 series · Five-task average | set size | Hurt |
| Use instruction-finetuned Chat-V2 formatting instead of the base model | Yi series · Five-task average | accuracy | Hurt |
| Use instruction-finetuned Chat-V1 or Chat-V2 | DeepSeek series · Five-task average | accuracy | Helped |
| Use instruction-finetuned Chat-V1 | DeepSeek-7B and DeepSeek-70B · Five-task average | set size | Hurt |
| Use instruction-finetuned Chat-V1 or Chat-V2 | Falcon series · Five-task average | set size | Hurt |
| Use APS instead of LAC as the conformal score function | Conformal prediction · Most tasks | set size | Hurt |
Surprising findings
- Higher accuracy does not necessarily imply lower uncertainty; e.g., Qwen-14B has much higher accuracy than Yi-6B on CI but larger set size under APS.
- Qwen-72B shows higher uncertainty than Qwen-14B on RC and DRS despite higher accuracy.
Limitations
- Tasks are reformulated as six-option multiple-choice questions, which may not reflect free-form generation behavior.
- Coverage guarantee is not maintained for free-form generation with sampling (TriviaQA setting).
- Closed-source models evaluated via sampling approximation of probabilities rather than logits.
Experimental setup
Datasets: MMLU CosmosQA HellaSwag HaluEval dialogue (HaluDial) HaluEval summarization (HaluSum) TriviaQA
Benchmarks: MMLU CosmosQA HellaSwag HaluDial HaluSum TriviaQA
Metrics: Accuracy Set Size Coverage Rate Expected Calibration Error
Baselines: entropy maximal predicted probability perplexity
Builds on: conformal prediction LAC APS
Closest papers by meaning
- An Empirical Analysis of Uncertainty in Large Language Model Evaluations (ICLR 2025 · similarity 0.93)
- Measuring what Matters: Construct Validity in Large Language Model Benchmarks (NeurIPS 2025 · similarity 0.92)
- Position: Don't Use the CLT in LLM Evals With Fewer Than a Few Hundred Datapoints (ICML 2025 · similarity 0.91)
- tinyBenchmarks: evaluating LLMs with fewer examples (ICML 2024 · similarity 0.91)
- Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities (NeurIPS 2024 · similarity 0.91)
- Large Language Models Must Be Taught to Know What They Don’t Know (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.