Research map / Reinforcement learning / RL from feedback
On the Bias of Group-Based Advantage Estimation
Fengkai Yang, Zherui Chen, Xiaohan Wang, Xiaodong Lu, Jiajun Chai, Guojun Yin, Wei Lin, Fuzhen Zhuang, Shuai Ma, deqing wang, Yaodong Yang, Yikun Ban
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes the theoretical bias of group-relative advantage estimation used in GRPO-style RLVR methods for LLM reasoning post-training.
Problem
Group-based advantage estimation (GRPO and variants) avoids learned critics in RLVR, but its theoretical properties are poorly understood.
Approach
Identifies a fundamental bias in group-relative advantage estimation for group-based RLVR methods; the abstract is truncated so the specific correction is not stated.
Experimental setup
Builds on: GRPO
Closest papers by meaning
- Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptation (NeurIPS 2026 · similarity 0.91)
- Conditional Advantage Estimation for Reinforcement Learning in Large Reasoning Models (ICLR 2026 · similarity 0.90)
- Leveraging Error Diversity in Group Rollouts for Reinforcement Learning (NeurIPS 2026 · similarity 0.90)
- Group-in-Group Policy Optimization for LLM Agent Training (NeurIPS 2025 · similarity 0.89)
- What are Key Factors for Updates in RL for LLM Reasoning? (NeurIPS 2026 · similarity 0.88)
- Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Exploration (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
