Research map / Large language models / Alignment and preferences
Online Rubrics Elicitation from Pairwise Comparisons
MohammadHossein Rezaei, Robert Vacareanu, Zihao Wang, Clinton Wang, Bing Liu, Yunzhong He, Afra Feyza Akyürek
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add online elicited criteria from pairwise comparisons (pi_ref control) to human rubrics | GRPO Qwen2.5-7B-Instruct · AlpacaEval win rate | win rate: 46.4 -> 54.0 | Improved |
| add online elicited criteria (pi_ref) to human rubrics | GRPO Qwen2.5-7B-Instruct · AlpacaEval | length-controlled win rate: 28.0 -> 31.5 | Improved |
| add online elicited criteria (pi_old) to human rubrics | GRPO Qwen2.5-7B-Instruct · Arena-Hard | win rate: 52.4 -> 56.5 | Improved |
| elicit criteria from pairwise comparison instead of pointwise (ablation) | OnlineRubrics · AlpacaEval win rate | win rate: 48.1 -> 54.0/55.0 | Improved |
| elicit criteria from pairwise comparison instead of pointwise (ablation) | OnlineRubrics · Arena-Hard | win rate: 51.1 -> 55.7/56.5 | Improved |
| use human-written rubrics instead of synthetic rubrics (ablation) | rubric RL · Generalist and Expert benchmarks | win rate | Improved |
| add universal requirements to offline rubrics versus OnlineRubrics (ablation) | rubric RL · Generalist and Expert benchmarks | multiple | Improved |
| add pointwise extraction to offline rubrics (ablation) | GRPO Qwen2.5-7B-Instruct · GPQA-Diamond | accuracy: 36.2 -> 33.6 | Hurt |
| use rubric-based rewards instead of LLM-judge Likert score (ablation) | GRPO Qwen2.5-7B-Instruct · AlpacaEval win rate | win rate: 42.2 -> 46.4 | Improved |
| increase number of response pairs from 1 to 8 for elicitation (ablation) | OnlineRubrics · AlpacaEval | win rate: 7-10% relative improvement | Improved |
| use lightweight extractor (GPT-4.1-nano) instead of o3-mini (ablation) | OnlineRubrics · benchmarks | performance | No clear effect |
| scale policy size 3B to 14B (ablation) | OnlineRubrics · rubric satisfaction gain over Offline Rubrics (Human) | pp gain: +1.3 pp -> +2.6 pp -> +3.2 pp | Improved |
Closest papers by meaning
- Step-wise Rubric Rewards for LLM Reasoning (NeurIPS 2026 · similarity 0.91)
- Reward Hacking in Rubric-Based Reinforcement Learning (NeurIPS 2026 · similarity 0.88)
- Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR (NeurIPS 2026 · similarity 0.87)
- Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for Open-Ended LLM Reasoning (ICML 2026 · similarity 0.86)
- Deep Research as Rubric (NeurIPS 2026 · similarity 0.86)
- Nudging the Boundaries of LLM Reasoning (ICLR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
