AI Paper Atlas

Research map / Large language models / Alignment and preferences

Pluralistic Leaderboards

Nika Haghtalab, Ariel D. Procaccia, Han Shao, Serena Lutong Wang, Kunhe Yang

ICML 2026 · Large language models › Alignment and preferences · approach: LLM-as-judge and feedback data

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
rank by Bradley-Terry scores instead of the local-stability ranking algorithmsLMArena leaderboard ranking · LMArena-based mixture of Mallows, top 20 models, phi=0.1stability ratio (gamma-hat)Hurt
use committee selection via iterated rounding (Algorithm 1) instead of top-k Bradley-Terry prefixcommittee selection · Synthetic mixture of Mallows, m=20stability ratio (gamma-hat)Improved
use top-ranked candidate of each Mallows center instead of Algorithm 1 as committeecommittee selection · Synthetic mixture of Mallows, m=20, phi=0.9stability ratio (gamma-hat)Hurt
use ranking via geometric checkpoints or committee monotonicity instead of Bradley-Terryleaderboard ranking · LMArena-based simulation, all phistability for all top-k prefixesImproved
use the heuristic committee-monotone ranking (Algorithm 3) instead of the checkpoint ranking (Algorithm 2) (ablation)pluralistic leaderboard ranking · LMArena-based simulationstability ratio (gamma-hat)Improved
increase user dispersion phi to 0.9 in simulation (ablation)Bradley-Terry ranking · LMArena-based simulationstability violationsImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy