Research map / Reinforcement learning / RL from feedback
Beyond Euclidean Clipping: Overcoming Exploration Collapse in LLM RL via Riemannian Isometric Policy Optimization
Zhicheng Cai, Xinyuan Guo, Hanlin Wu, Mingxuan Wang, Wei-Ying Ma, Ya-Qin Zhang, Hao Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace GRPO clipping with RIPO isometric clipping | GRPO · 7 math benchmarks average, Qwen3-8B-Base | Avg@8 (%): 28.5 -> 38.5 | Improved |
| replace GRPO clipping with RIPO | GRPO · 7 math benchmarks average, Qwen3-1.7B-Base | Avg@8 (%): 11.2 -> 15.4 | Improved |
| replace GRPO clipping with RIPO | GRPO · 7 math benchmarks average, Llama3.2-3B-Instruct | Avg@8 (%): 6.4 -> 8.6 | Improved |
| replace GRPO clipping with RIPO | GRPO · 7 math benchmarks average, Qwen3-4B-Base | Avg@8 (%): 25.7 -> 30.1 | Improved |
| replace DCPO clipping with RIPO | DCPO · 7 math benchmarks average, Qwen3-8B-Base | Avg@8 (%): 34.5 -> 38.5 | Improved |
| replace DCPO clipping with RIPO | DCPO · 7 math benchmarks average, Llama3.2-3B-Instruct | Avg@8 (%): 6.9 -> 8.6 | Improved |
| use symmetric delta 0.05 instead of 0.02 (ablation) | RIPO · AIME24, Qwen3-8B-Base | Avg@8 (%): 40.8 -> 43.8 | Improved |
| make delta_high larger than delta_low (0.05/0.02 vs 0.05/0.05) (ablation) | RIPO · AIME24, Qwen3-8B-Base | Avg@8 (%): 43.8 -> 28.8 | Hurt |
| use asymmetric delta_low 0.08, delta_high 0.04 instead of symmetric 0.08 (ablation) | RIPO · AIME24, Qwen3-8B-Base | Avg@8 (%): 42.1 -> 27.9 | Hurt |
| vary symmetric delta from 0.02 to 0.08 (ablation) | RIPO · AIME24, Qwen3-8B-Base | Avg@8 (%): 40.8, 43.8, 42.1 | No clear effect |
| use RIPO-Clip instead of PPO-Clip in PPO | PPO · GSM8K, Qwen2.5-0.5B-Instruct | Avg@1 (%): 58.0 -> 61.1 | Improved |
| use RIPO-Clip instead of PPO-Clip in PPO | PPO · GSM8K, Qwen2.5-14B-Instruct | Avg@1 (%): 93.2 -> 94.4 | Improved |
Closest papers by meaning
- Parameter Exploration for RLVR via Variational Learning (NeurIPS 2026 · similarity 0.86)
- What are Key Factors for Updates in RL for LLM Reasoning? (NeurIPS 2026 · similarity 0.86)
- A Differential and Pointwise Control Approach to Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
- SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMs (ICML 2026 · similarity 0.86)
- Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstraction (ICML 2024 · similarity 0.86)
- LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
