AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

G$^2$RPO: Geometric GRPO; Escaping LLM's Reasoning Rut to Break Accuracy--Entropy Trade-off

Ali Rad, Khashayar Filom, Darioush Keivan, Peyman Mohajerin Esfahani, Ehsan Kamalinejad

ICML 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Diagnoses diversity collapse in GRPO as a dynamical flow on the probability simplex and proposes G²RPO (Geometric GRPO) to escape the resulting reasoning rut and improve the accuracy-entropy trade-off.

Problem

GRPO-style RLVR for large reasoning models often suffers diversity collapse (entropy loss), limiting exploration and creating an accuracy-entropy trade-off.

Approach

Formalizes GRPO as a dynamical flow on the probability simplex under mode-based coarse-graining of rollouts, showing it induces a geometric rut that suppresses diversity; proposes a geometry-aware modification (G²RPO) to escape it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add inverse-probability granularity bonus (G2RPO) to GRPO advantagesGRPO · AIME 2024 / 2025pass@1: +1.4 to +7.9 points vs GRPOImproved
add G2RPO bonus to GRPOQwen3-14B-Base · AIME 2024pass@1: 45.9 -> 53.8Improved
add G2RPO bonus to GRPODeepSeek-R1-Distill-Qwen-7B · AIME 2025pass@1: 46.7 -> 48.9Improved
add G2RPO bonus to GRPODeepSeek-R1-Distill-Qwen-7B · mode diversityAvg K (correct clusters): 1.97 -> 6.40Improved
train with plain GRPO instead of staying at base modelQwen3-14B-Base · mode diversityAvg K (correct clusters): 3.36 -> 1.45Hurt
increase gain lambda from 1.5 to 3 (ablation)G2RPO · mode coveragegood modes discovered: +92% over lambda=1.5Improved
increase gain lambda without neutrality correction (ablation)G2RPO · bad-mass learningincorrect fraction p-hat decayHurt
add neutrality correction (bad-mass offset) (ablation)G2RPO · bad-mass learningp-hat trajectoryImproved
swap sentence encoder and DBSCAN scale (ablation)G2RPO · DeepSeek-7B pass@30pass@30: 78% unchangedNo clear effect
add G2RPO bonus to GRPOGRPO · AIME 2025 pass@k, k in {2,4,16,30}pass@k: +1.7 to +4.3 (Qwen3-14B), +2.0 to +4.9 (DeepSeek-7B)Improved
add moderate KL regularization (ablation)G2RPO · diversitymode diversity advantageMixed

Experimental setup

Metrics: accuracy entropy

Baselines: GRPO

Builds on: GRPO

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy