Research map / Reinforcement learning / RL from feedback
Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Training
Ran Xu, Tianci Liu, Zihan Dong, Tony Yu, Ilgee Hong, Carl Yang, Linjun Zhang, Tuo Zhao, Haoyu Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| update the judge first, then the rubric generator (ablation) | Rubric-ARM · Reward-modeling benchmarks (avg of 9) | average accuracy: 72.4 -> 74.8 | Improved |
| update judge first (voting@5) (ablation) | Rubric-ARM · Reward-modeling benchmarks (avg of 9) | average accuracy: 74.9 -> 76.2 | Improved |
| add format reward (ablation) | Rubric-ARM · Reward-modeling benchmarks (avg of 9) | average accuracy: 72.6 -> 74.8 | Improved |
| add format reward (voting@5) (ablation) | Rubric-ARM · Reward-modeling benchmarks (avg of 9) | average accuracy: 75.5 -> 76.2 | Improved |
| add format reward on RewardBench2 Precise IF (ablation) | Rubric-ARM · RewardBench2 Precise IF | accuracy: 25.6 -> 41.9 | Improved |
| add format reward on RewardBench Chat (ablation) | Rubric-ARM · RewardBench Chat | accuracy: 89.8 -> 89.4 | No clear effect |
| alternating RL on rubric generator and judge instead of SFT-only (Rubric-RM) | Rubric-ARM · Reward-modeling benchmarks (avg of 9) | average accuracy: 70.1 -> 74.8 | Improved |
| majority voting over 5 judges | Rubric-ARM · Reward-modeling benchmarks (avg of 9) | average accuracy: 74.8 -> 76.2 | Improved |
| use Rubric-ARM instead of Rubric-RM as DPO reward for Qwen2.5-7B-Instruct | Rubric-ARM · IFEval | average: 79.5 -> 80.4 | Improved |
| iterative DPO over single-round DPO with Rubric-ARM | Rubric-ARM · Arena-Hard + AlpacaEval | average: 51.7 -> 53.4 | Improved |
| iterative DPO vs DPO with Rubric-RM on WildBench | Rubric-RM · WildBench | WB Score: 53.6 -> 54.0 | Mixed |
| use Rubric-ARM instead of RM-R1 as GRPO reward | Rubric-ARM · Online RL avg | average: 52.3 -> 55.4 | Improved |
Closest papers by meaning
- Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR (NeurIPS 2026 · similarity 0.90)
- Reward Hacking in Rubric-Based Reinforcement Learning (NeurIPS 2026 · similarity 0.90)
- Step-wise Rubric Rewards for LLM Reasoning (NeurIPS 2026 · similarity 0.89)
- LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL (NeurIPS 2026 · similarity 0.88)
- Post-hoc Reward Calibration: A Case Study on Length Bias (ICLR 2025 · similarity 0.88)
- Deep Research as Rubric (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
