Research map / Reinforcement learning / RL from feedback
GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning
Xiangxiang Chu, Hailang Huang, Xiao Zhang, Fei Wei, Yong Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace GRPO loss with plain policy gradient and group-mean reward (std normalization), no KL/reference (ablation) | GPG · Math avg (5 benchmarks, ablation on MATH-lighteval training) | average accuracy: 43.7 -> 45.3 | Improved |
| remove std reward normalization (Fnorm=1) from GPG (ablation) | GPG · Math avg (5 benchmarks, ablation on MATH-lighteval training) | average accuracy: 45.3 -> 43.9 | Hurt |
| add accurate gradient estimation (alpha=B/(B-M)) to Fnorm=1 GPG (ablation) | GPG · Math avg (5 benchmarks, ablation on MATH-lighteval training) | average accuracy: 43.9 -> 47.8 | Improved |
| add valid-sample threshold beta_th=0.6 with resampling (ablation) | GPG · Math avg (5 benchmarks, ablation on MATH-lighteval training) | average accuracy: 47.8 -> 48.3 | Improved |
| add alpha correction to std-normalized GPG (ablation) | GPG · Math avg (5 benchmarks, ablation on MATH-lighteval training) | average accuracy: 45.3 -> 44.1 | Hurt |
| add a KL distribution constraint to GPG (ablation) | GPG · Math reasoning | accuracy | Hurt |
| increase group size from 2 to 16 (ablation) | GPG · Math reasoning | average accuracy | Improved |
| use GPG instead of GRPO on 7B base | Qwen2.5-Math-7B · Math avg (5 benchmarks, ablation on MATH-lighteval training) | average accuracy: 43.7 -> 48.3 | Improved |
| use GPG instead of DAPO | Qwen2.5-Math-7B · Math avg, DAPO comparison | average accuracy: 56.0 -> 57.7 | Improved |
| use GPG instead of GRPO for geometry reasoning | Qwen2.5-VL-3B-Instruct · GEOQA test | accuracy: 47.48 -> 51.33 | Improved |
| use GPG instead of GRPO on CV-Bench | Qwen2-VL-2B · CV-Bench | total accuracy: 59.47 -> 76.15 | Improved |
| use GPG instead of GRPO for LISA grounding | Qwen2-VL-2B · LISA | mIoU: 37.6 -> 51.8 | Improved |
Closest papers by meaning
- Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning (ICLR 2026 · similarity 0.91)
- All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models (CVPR 2026 · similarity 0.91)
- Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models (NeurIPS 2025 · similarity 0.91)
- Accelerating RL for LLM Reasoning with Optimal Advantage Regression (NeurIPS 2025 · similarity 0.90)
- Advancing LLM Reasoning with Natural Language and Numerical Feedback (ICML 2026 · similarity 0.90)
- BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
