Research map / Learning theory and algorithms / Prediction and decision losses
Strategic Candidacy in Generative AI Arenas
Chris Hays, Rachel Li, Bailey Flanigan, Manish Raghavan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| submit one clone of a model | Status Quo (Bradley-Terry MLE) arena ranking · Arena simulation (Jan 2026 snapshot, all arenas) | rank positions gained: some models move up >= 7 positions with just one clone | Improved |
| submit more clones (1 to 5) | Status Quo arena ranking · Arena simulation, all arenas | rank positions gained | Improved |
| fewer votes per model pair (s <= 30: Coding, Expert, Multiturn) | Status Quo arena ranking · Arena simulation, cloning benefit | rank positions gained via clones | Improved |
| replace Status Quo with You-Rank-We-Rank (producer self-rankings, min score correction) | Arena ranking mechanism · Arena simulation, correct producer rankings | rank positions gained via cloning: essentially zero gains in rank via cloning | Improved |
| replace Status Quo with You-Rank-We-Rank under correct producer rankings | Arena ranking mechanism · Arena simulation, six arenas, all clone counts | Kendall-Tau distance to ground truth: lower distance, often by many tens of swaps | Improved |
| add noise to producer self-rankings (ablation) | You-Rank-We-Rank · Arena simulation, Kendall-Tau noise 10% to >100% of list length | accuracy advantage over Status Quo: advantage diminishes; below zero only when noise is on order of number of models | Hurt |
| YRWR clone benefit for mid-ranked models in tight clusters | You-Rank-We-Rank · Arena simulation, Appendix B | rank change from one clone: up to around two positions for some models | No clear effect |
Closest papers by meaning
- The Leaderboard Illusion (NeurIPS 2025 · similarity 0.88)
- Improving Your Model Ranking on Chatbot Arena by Vote Rigging (ICML 2025 · similarity 0.87)
- K-Sort Arena: Efficient and Reliable Benchmarking for Generative Models via K-wise Human Preferences (CVPR 2025 · similarity 0.87)
- Pricing and Competition for Generative AI (NeurIPS 2024 · similarity 0.86)
- Position: AI Competitions Provide the Gold Standard for Empirical Rigor in GenAI Evaluation (ICML 2025 · similarity 0.86)
- A Statistical Framework for Ranking LLM-based Chatbots (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
