Research map / Reinforcement learning / RL from feedback
From Absolute to Relative: Rethinking Reward Shaping in Group-Based Reinforcement Learning
Wenzhe Niu, Wei He, Zongxia Xie, Jinpeng Ou, Huichuan Fan, Yuchen Ge, Yanru Sun, Ziyin Wang, Yizhao Sun, Chengshun Shi, Jiuchong Gao, Jinghua Hao and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add correctness to hierarchical re-ranking (ablation) | RLRR · Math (6 benchmarks) | average accuracy (%): 45.7 -> 47.1 | Improved |
| add length bins to hierarchical re-ranking (ablation) | RLRR · Math (6 benchmarks) | average accuracy (%): 47.2 -> 47.1 | Hurt |
| add length bins to hierarchical re-ranking (ablation) | RLRR · Math (6 benchmarks) | average response length (tokens): 6317 -> 6013 | Improved |
| replace random ranking with Ranking RM within correctness groups (ablation) | RLRR · Math avg (AIME24, AIME25, Minerva, AMC) | average accuracy (%): 37.5 -> 38.0 | Improved |
| add correctness re-ranking with random order to GRPO (ablation) | GRPO · Math avg (AIME24, AIME25, Minerva, AMC) | average accuracy (%): 36.3 -> 37.5 | Improved |
| replace GRPO with RLRR (HRR) on 1.5B | DeepSeek-R1-Distill-Qwen-1.5B · Logic reasoning avg | average accuracy (%): 12.5 -> 24.8 | Improved |
| replace GRPO with RLRR (HRR) on 8B | DeepSeek-R1-Distill-Llama-8B · Math reasoning avg | average accuracy (%): 62.9 -> 65.9 | Improved |
| use PRR instead of HRR (no rule-based supervision) (ablation) | RLRR · Math reasoning avg, 8B | average accuracy (%): 65.9 -> 64.1 | Hurt |
| add RLRR to CISPO | CISPO · Math avg | average accuracy (%): 36.1 -> 37.1 | Improved |
| add RLRR to RLOO | RLOO · Math avg | average accuracy (%): 36.1 -> 37.1 | Improved |
| add RLRR to GRPO at group size 4 | GRPO · Math avg, G=4 | average accuracy (%): 54.1 -> 54.5 | Improved |
| add RLRR to GRPO at group size 16 | GRPO · Math avg, G=16 | average accuracy (%): 55.4 -> 55.4 | No clear effect |
Closest papers by meaning
- SIRAS: Sibling-Relative Advantage Shaping for Reinforcement Learning from Verifiable Rewards (NeurIPS 2026 · similarity 0.91)
- On the Bias of Group-Based Advantage Estimation (NeurIPS 2026 · similarity 0.91)
- ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking (ICML 2026 · similarity 0.91)
- Group-in-Group Policy Optimization for LLM Agent Training (NeurIPS 2025 · similarity 0.89)
- Listwise Policy Optimization: Group-based RLVR as Target-Projection on the LLM Response Simplex (NeurIPS 2026 · similarity 0.88)
- Revisiting On-policy Adversarial Black-Box Distillation: Calibrating Groupwise Reward Geometry for Effective Advantage Construction (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
