Research map / Large language models / Alignment and preferences
Prediction-Powered Ranking of Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a prediction-powered inference framework that builds rank-sets with asymptotic coverage guarantees for ranking LLMs from a small set of human pairwise comparisons plus a large set of strong-LLM judgments.
Problem
Rankings of LLMs built from pairwise comparisons by a strong LLM judge may not match human preference rankings, and no existing method quantifies the resulting ranking uncertainty.
Approach
Combine a small human-labeled pairwise set with a large LLM-labeled set via prediction-powered inference to get a confidence ellipsoid over win-rates, then derive rank-sets as the ranking positions not excluded by hyperplane separation from that ellipsoid, with coverage of the human-consistent ranking.
Key results
- LLM-only rank-sets (GPT-4) achieve much lower baseline intersection probability than human-only or PPR rank-sets, and the top-ranked position differs from the baseline for 7 of 12 models.
- PPR with GPT-4 (n=990, alpha=0.05) differs from the baseline top position for only 3 of 12 models, with better size/intersection trade-off than human-only rank-sets.
- In synthetic experiments with alpha=0.1 and N+n=50,000, LLM-only rank-sets reach coverage of 0.38, 0.13, and 0.0 for noise levels 0.05, 0.1, 0.3, while PPR achieves coverage near the 1-alpha target.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine strong-LLM comparisons with a small set of human comparisons instead of using strong-LLM comparisons alone | Prediction-powered ranking · LMSYS Chatbot Arena | baseline intersection probability | Helped |
| use GPT-4 comparisons rather than Claude 3 or GPT-3.5 comparisons in prediction-powered ranking | Prediction-powered ranking · LMSYS Chatbot Arena | baseline intersection probability | Helped |
| increase the number of human pairwise comparisons | Prediction-powered ranking · LMSYS Chatbot Arena | trade-off between rank-set size and baseline intersection probability | Helped |
| use only strong-LLM pairwise comparisons rather than human-only comparisons | LLM-only ranking · LMSYS Chatbot Arena | baseline intersection probability | Hurt |
| use only GPT-4 comparisons rather than prediction-powered GPT-4 ranking | GPT-4 ranking · LMSYS Chatbot Arena | agreement of the most probable ranking position with the human baseline: 7 out of 12 vs 3 out of 12 | Hurt |
| increase the number of human comparisons n | Prediction-powered ranking · Synthetic experiments | coverage probability | Helped |
| use a strong LLM more aligned with human preferences (smaller noise u) | Prediction-powered ranking · Synthetic experiments | coverage probability | Helped |
| combine human preferences with strong-LLM preferences rather than relying only on strong-LLM preferences | Ranking methods · Synthetic experiments | rank-biased overlap | Helped |
| increase the number of human pairwise comparisons n | Ranking methods · Synthetic experiments | rank-biased overlap | Helped |
| use only strong-LLM comparisons rather than human-only or prediction-powered comparisons | LLM-only ranking · LMSYS Chatbot Arena | baseline coverage probability | Hurt |
| use prediction-powered ranking rather than human-only ranking | Prediction-powered ranking · LMSYS Chatbot Arena | baseline coverage probability | No clear effect |
Surprising findings
- Rank-sets built only from strong LLM judgments often are inconsistent with human-preference rankings, with coverage near zero in the less aligned synthetic setting.
- GPT-4 clearly outperforms GPT-3.5 and Claude 3 as a judge in the rank-set quality metrics.
Limitations
- Assumes i.i.d. queries and comparisons and truthful human labels; no distribution shift or strategic behavior handling.
- Guarantees are asymptotic rather than finite-sample; rank-sets can be large and overlapping with limited data.
- Evaluation limited to one platform (LMSYS Chatbot Arena) and three judge LLMs; quality metrics are proxies since true rank-sets are unknown on real data.
Experimental setup
Datasets: LMSYS Chatbot Arena
Benchmarks: LMSYS Chatbot Arena pairwise comparisons (12 LLMs)
Metrics: rank-set size baseline intersection probability baseline coverage probability coverage probability rank-biased overlap
Baselines: human-only rank-sets LLM-only rank-sets (GPT-4, GPT-3.5, Claude 3) Elo Bradley-Terry
Builds on: prediction-powered inference win-rate ranking Neuhof and Benjamini rank-set construction
Closest papers by meaning
- Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models (ICLR 2024 · similarity 0.91)
- A Statistical Framework for Ranking LLM-based Chatbots (ICLR 2025 · similarity 0.90)
- Improving Context-Aware Preference Modeling for Language Models (NeurIPS 2024 · similarity 0.89)
- Preference Ranking Optimization for Human Alignment (AAAI 2024 · similarity 0.89)
- Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes (ICML 2025 · similarity 0.89)
- Aligning Language Model Benchmarks with Pairwise Preferences (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.