AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Training

Junkai Zhang, Zihao Wang, Lin Gui, Swarnashree Mysore Sathyendra, Jaehwan Jeong, Victor Veitch, Wei Wang, Yunzhong He, Bing Liu, Lifeng Jin

ICLR 2026 · Reinforcement learning › RL from feedback · approach: Reward modeling versus direct optimization

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
refine rubrics with a great pair instead of a good pair (ablation)rubric RFT (Qwen3-8B-Base) · Medical-o1 win ratewin rate: 22.4 -> 26.5Improved
refine rubrics with one great pair vs prompt-only rubric (ablation)rubric RFT (Qwen3-8B-Base) · Finance win ratewin rate: 37.2 -> 42.2Improved
iterate refinement with 4 great pairs instead of 1 (ablation)rubric RFT (Qwen3-8B-Base) · Medical-o1 win ratewin rate: 26.5 -> 31.4Improved
add diverse candidate responses to 4 great pairs (ablation)rubric RFT (Qwen3-8B-Base) · Medical-o1 win ratewin rate: 31.4 -> 34.4Improved
add diverse candidate responses to 4 great pairs (ablation)rubric RFT (Qwen3-8B-Base) · Medical-o1 HealthBenchscore: 0.3348 -> 0.3513Improved
refine rubric with one good pair vs prompt-only (ablation)rubric RFT (Qwen3-8B-Base) · Medical-o1 HealthBenchscore: 0.3004 -> 0.2912No clear effect
refine with great pairs instead of good pairs (ablation)rubric accuracy · high-reward region pairsaccuracy: 42.2% -> 45.8%Improved
refine with good pairs instead of great pairs (ablation)rubric accuracy · low-reward region pairsaccuracy: 66.7% -> 67.9%Improved
add diverse responses to 4 great pairs (ablation)rubric accuracy · high-reward region pairsaccuracy: 49.2% -> 47.9%Hurt
train on rubrics refined with great diverse pairs rather than initial rubrics, long training (ablation)rubric RFT · health and finance extended trainingwin rate and benchmark scoreImproved
Bradley-Terry reward model trained on off-policy great responses vs on-policy (ablation)BT reward model (Llama-3.1-8B) · Health domainwin rate: 26.8% -> 22.4%Hurt
add off-policy to on-policy responses for BT reward model (ablation)BT reward model (Llama-3.1-8B) · Health domainHealthBench: 0.3036 -> 0.3032No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy