Research map / Reinforcement learning / RL from feedback
The Capability Frontier of GRPO
Rithvik Redrouthu, Jackson Stokes
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Argues that GRPO's rollout group size G acts as a difficulty threshold, since under binary verifier rewards prompts with no success/failure mix in their group yield no learning signal.
Problem
GRPO is widely used to post-train reasoning LMs, but its rollout group size G is usually treated only as a compute hyperparameter, with unclear effects on which prompts yield learning signal.
Approach
Frames G as setting a difficulty threshold: under binary verifier rewards, a group yields a learning signal only if it contains both a success and a failure, so prompts whose success rate is too low or too high within G rollouts produce no gradient, shaping the capability frontier of GRPO.
Experimental setup
Builds on: GRPO
Closest papers by meaning
- WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning (ICML 2026 · similarity 0.91)
- Learn from the Gap: Differential-Aware Advantage Pruning with Adaptive Rollout Sampling for GRPO (NeurIPS 2026 · similarity 0.91)
- GVPO: Group Variance Policy Optimization for Large Language Model Post-Training (NeurIPS 2025 · similarity 0.89)
- TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback (NeurIPS 2026 · similarity 0.88)
- On the Effect of Negative Gradient in Group Relative Deep Reinforcement Optimization (NeurIPS 2025 · similarity 0.88)
- CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
