Research map / Reinforcement learning / RL from feedback
DRPO: Efficient Reasoning via Decoupled Reward Policy Optimization
Gang Li, Yan Chen, Ming Lin, Tianbao Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add length reward to GRPO-style advantage (RLOO-LP, ALP, HAPO) | GRPO-style methods · math average, 7B | pass@1 and AES: RLOO-LP: 38% length reduction with 7.1% performance loss | Hurt |
| decouple length reward by normalizing it only within correct rollouts (DRPO) | DisCO · math average, 7B | reasoning length and performance: 3053 -> 1502 tokens (51% reduction), 2.6% performance loss | Improved |
| use DRPO instead of HAPO | DeepSeek-R1-Distill-Qwen-7B · math average | pass@1: 0.596 -> 0.714 | Improved |
| use DRPO instead of ALP | DeepSeek-R1-Distill-Qwen-1.5B · math average | AES: -0.387 -> 0.178 | Improved |
| use DRPO instead of RLOO-LP | DeepSeek-R1-Distill-Llama-8B · math average | AES: 0.251 -> 0.297 | Improved |
| apply DRPO on easy questions | DeepSeek-R1-Distill-Qwen-1.5B · GSM8K | length and accuracy: 1563 -> 356 tokens (77.2% reduction), -1.1% accuracy | Improved |
| decrease lambda from 0.5 to 0.1 in DRPO (ablation) | DRPO 7B · math average | AES: -0.007 -> 0.249 | Improved |
| decrease lambda from 0.5 to 0.1 in DRPO (ablation) | DRPO 1.5B · AIME | AES: -0.143 -> -0.729 | Hurt |
| use concave or cosine length reward instead of linear in DRPO (ablation) | DRPO 1.5B · math average | accuracy vs length trade-off | Mixed |
| apply DRPO to non-math reasoning | DeepSeek-R1-Distill-Qwen-1.5B · K&K logic puzzles | length and accuracy: 2095 -> 1400 tokens; accuracy 0.972 -> 0.974 | Improved |
Closest papers by meaning
- Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts (NeurIPS 2025 · similarity 0.90)
- DisCO: Reinforcing Large Reasoning Models with Discriminative Constrained Optimization (NeurIPS 2025 · similarity 0.90)
- DAPO : Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage-Based Policy Optimization (NeurIPS 2025 · similarity 0.90)
- S-GRPO: Early Exit via Reinforcement Learning in Reasoning Models (NeurIPS 2025 · similarity 0.90)
- WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning (ICML 2026 · similarity 0.90)
- Fast-Slow Thinking GRPO for Large Vision-Language Model Reasoning (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
