Research map / Reinforcement learning / RL from feedback
Geometric-Mean Policy Optimization
Yuzhong Zhao, Yue Liu, Junpeng Liu, Jingye Chen, Xun Wu, Yaru Hao, Tengchao Lv, Shaohan Huang, Lei Cui, Qixiang Ye, Fang Wan, Furu Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace arithmetic mean with geometric mean of token rewards (GMPO vs GRPO) | GRPO, Qwen2.5-Math-7B · 5 math benchmarks avg | Pass@1 (%): 51.2 -> 52.7 | Improved |
| GMPO vs GRPO | DeepSeek-R1-Distill-Qwen-7B · 5 math benchmarks avg | Pass@1 (%): 59.3 -> 63.4 | Improved |
| GMPO vs GRPO | Qwen2.5-Math-1.5B · 5 math benchmarks avg | Pass@1 (%): 42.5 -> 43.9 | Improved |
| GMPO vs GRPO | Qwen2.5-VL-Instruct-7B · Geometry3K | Pass@1 (%): 53.3 -> 54.7 | Improved |
| GMPO vs GRPO | Qwen3-32B MoE · MATH500 | Pass@1 (%): 94.6 -> 96.7 | Improved |
| add 1/|o| length normalization to the geometric product (ablation) | GMPO · 5 math benchmarks avg | Pass@1 (%): 52.0 -> 52.7 | Improved |
| add clipping to GMPO (ablation) | GMPO · 5 math benchmarks avg | Pass@1 (%): 52.3 -> 52.7 | Improved |
| token-level clipping instead of sequence-level clipping (ablation) | GMPO · 5 math benchmarks avg | Pass@1 (%): 52.6 (seq-clip) vs 52.7 | Mixed |
| widen clip range to (e^-0.4, e^0.4) from (e^-0.2, e^0.2) (ablation) | GMPO · 5 math benchmarks avg | Pass@1 (%): 52.4 -> 52.7 | Improved |
| widen clip range further to (e^-0.8, e^0.8) (ablation) | GMPO · 5 math benchmarks avg | Pass@1 (%): 52.7 -> 52.1 | Hurt |
Closest papers by meaning
- C-GRPO: Conformal Group Relative Policy Optimization (NeurIPS 2026 · similarity 0.91)
- GTPO and GRPO-S: Token and Sequence-Level Reward Shaping with Policy Entropy (ICML 2026 · similarity 0.91)
- GVPO: Group Variance Policy Optimization for Large Language Model Post-Training (NeurIPS 2025 · similarity 0.90)
- TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback (NeurIPS 2026 · similarity 0.90)
- WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning (ICML 2026 · similarity 0.90)
- Multi-Task GRPO: Reliable LLM Reasoning Across Tasks (ICML 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
