Research map / Learning theory and algorithms / Prediction and decision losses
Robust AI Evaluation through Maximal Lotteries
Hadi Khalaf, Serena Lutong Wang, Daniel Halpern, Itai Shapira, Flavio Calmon, Ariel D. Procaccia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase robustness radius rho from 0 to 1 in the robust lottery (ablation) | Robust lottery · LMArena held-out votes | train-test win rate gap: roughly 6% -> about 2% | Improved |
| increase robustness radius rho (ablation) | Robust lottery · LMArena and HUMAINE worst-group win rate | worst-group win rate on held-out votes | Improved |
| increase robustness radius rho (ablation) | Robust lottery · LMArena and HUMAINE best-performing groups | best-group win rate | Hurt |
Closest papers by meaning
- Prediction-Powered Ranking of Large Language Models (NeurIPS 2024 · similarity 0.89)
- Strong Preferences Affect the Robustness of Preference Models and Value Alignment (ICLR 2025 · similarity 0.87)
- Inverse Constitutional AI: Compressing Preferences into Principles (ICLR 2025 · similarity 0.87)
- HPS: Hard Preference Sampling for Human Preference Alignment (ICML 2025 · similarity 0.87)
- Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences? (NeurIPS 2025 · similarity 0.87)
- Aligning with Logic: Measuring, Evaluating and Improving Logical Preference Consistency in Large Language Models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
