Research map / Reinforcement learning / RL from feedback
XRPO: Pushing the Limits of GRPO with Targeted Exploration and Exploitation
Udbhav Bamba, Minghao Fang, Yifan Yu, Haizhong Zheng, Fan Lai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ICL seeding for zero-reward prompts (ablation) | XRPO · Qwen3-1.7B, 4 math benchmarks avg | cons@32: 43.33 -> 47.50 | Improved |
| add novelty-guided advantage sharpening (ablation) | XRPO · Qwen3-1.7B, 4 math benchmarks avg | pass@1: 34.35 -> 37.81 | Improved |
| add hierarchical rollout planning over uniform allocation (ablation) | XRPO · Qwen3-1.7B, 4 math benchmarks avg | pass@1: 35.10 -> 37.81 | Improved |
| use XRPO instead of GSPO | GSPO · Qwen3-1.7B, AIME'25 | pass@1: 33.23 -> 35.72 | Improved |
| use XRPO instead of GRPO | GRPO · Llama-3.2-3B, BRUMO'25 | pass@1: 4.79 -> 3.54 | Hurt |
| use XRPO instead of GRPO | GRPO · Llama-3.2-3B, avg of 3 benchmarks | pass@1: 4.17 -> 4.48 | Improved |
| add test-time ICL on top of XRPO (ablation) | XRPO · Qwen2.5-7B-Instruct | pass@1: 12.05 -> 11.77 | Hurt |
| use strong embedding retrieval for ICL vs no ICL (ablation) | ICL seeding · DAPO-Math-17k, Qwen2.5-7B | accuracy: 18.6 -> 24.5 | Improved |
| use random ICL retrieval vs no ICL (ablation) | ICL seeding · DAPO-Math-17k, Qwen3-1.7B | accuracy: 37.7 -> 38.12 | Improved |
| truncate 20% of ICL exemplars (ablation) | ICL seeding · Qwen3-1.7B | accuracy: 41.5 -> 39.0 | Hurt |
| use uncertainty-based allocation instead of difficulty-based CurES | XRPO · Qwen3-1.7B, avg of 3 benchmarks | cons@32: 32.22 -> 45.56 | Improved |
| combine XRPO with GSPO | GSPO · Qwen3-1.7B, HMMT'25 | pass@1: 21.2 -> 22.8 | Improved |
Closest papers by meaning
- Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts (NeurIPS 2025 · similarity 0.90)
- Stratified GRPO: Handling Structural Heterogeneity in Reinforcement Learning of LLM Search Agents (ICML 2026 · similarity 0.89)
- DAPO : Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage-Based Policy Optimization (NeurIPS 2025 · similarity 0.89)
- GVPO: Group Variance Policy Optimization for Large Language Model Post-Training (NeurIPS 2025 · similarity 0.89)
- Advancing LLM Reasoning with Natural Language and Numerical Feedback (ICML 2026 · similarity 0.89)
- DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
