Research map / Reinforcement learning / RL from feedback
Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward
Xinyu Tang, Zhenduo Zhang, Yurou Liu, Xin Zhao, zujie wen, Zhiqiang Zhang, JUN ZHOU
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PageRank-weighted DPP pruning (ablation) | DEPO (DeepSeek-R1-Distill-Qwen-7B) · AIME25 | accuracy: 50.0 -> 50.9 | Improved |
| add difficulty-aware normal-distribution sampling (ablation) | DEPO (DeepSeek-R1-Distill-Qwen-7B) · AIME25 | accuracy: 47.8 -> 50.9 | Improved |
| add explorability-based rollout selection instead of random (ablation) | DEPO (DeepSeek-R1-Distill-Qwen-7B) · AIME25 | accuracy: 45.3 -> 50.9 | Improved |
| add replay of under-explored samples (ablation) | DEPO (DeepSeek-R1-Distill-Qwen-7B) · AIME25 | accuracy: 48.4 -> 50.9 | Improved |
| add replay of under-explored samples (ablation) | DEPO (DeepSeek-R1-Distill-Qwen-7B) · MATH500 | accuracy: 95.2 -> 95.9 | Improved |
| use DEPO offline selection instead of Learnalign | GRPO on 20% DAPO-Math (DeepSeek-R1-Distill-Qwen-7B) · five-benchmark average | average accuracy: 58.7 -> 61.4 | Improved |
| use explorability rollout pruning instead of GRESO | DEPO offline subset (DeepSeek-R1-Distill-Qwen-7B) · five-benchmark average | average accuracy: 58.1 -> 61.1 | Improved |
| use random rollout pruning (40% rollouts) instead of full rollouts | DEPO offline subset (DeepSeek-R1-Distill-Qwen-7B) · five-benchmark average | average accuracy: 61.4 -> 56.7 | Hurt |
| online DEPO pruning vs full rollouts on DEPO subset | DeepSeek-R1-Distill-Qwen-7B · five-benchmark average | average accuracy: 61.4 -> 61.1 | No clear effect |
| DEPO offline on 20% data vs full dataset training | DeepSeek-R1-Distill-Llama-8B · five-benchmark average | average accuracy: 58.2 -> 57.9 | No clear effect |
| use mean difficulty 0.75 instead of medium (ablation) | difficulty-aware sampling · convergence performance | accuracy | Hurt |
| use mean difficulty 0.25 instead of medium (ablation) | difficulty-aware sampling · final performance | accuracy | Mixed |
Closest papers by meaning
- Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.91)
- Beyond Verifiable Rewards: Scaling Reinforcement Learning in Language Models to Unverifiable Data (NeurIPS 2025 · similarity 0.91)
- Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards (ICML 2026 · similarity 0.90)
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale (NeurIPS 2025 · similarity 0.90)
- Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor’s Internal States (NeurIPS 2026 · similarity 0.90)
- Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
