Research map / Reinforcement learning / RL from feedback
Learning More with Less: A Dynamic Dual-Level Down-Sampling Framework for Efficient Policy Optimization
Chao Wang, Tao Yang, Hongtao Tian, Yunsheng Shi, Qiyao Ma, XiaotaoLiu, Ting Yao, Wenbo Ding
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add sample-level advantage-variance down-sampling (D1S) (ablation) | GRPO · 7 math benchmarks | Pass@8 average: 53.1 -> 54.2 | Improved |
| add cross-group selection to D1S (ablation) | GRPO · 7 math benchmarks (Qwen2.5-Math-7B) | Pass@1 average: 31.3 -> 34.1 | Improved |
| add token-level entropy-advantage selection (D2S) without schedule (ablation) | GRPO · 7 math benchmarks (Qwen2.5-Math-7B) | Pass@8 average: 54.2 -> 54.1 | No clear effect |
| add dynamic down-sampling schedule (D3S vs D2S) (ablation) | GRPO · 7 math benchmarks (Qwen2.5-Math-7B) | Pass@8 average: 54.1 -> 56.8 | Improved |
| add D3S to GRPO | Qwen2.5-Math-7B · 7 math benchmarks | Pass@1 average: 29.8 -> 34.3 | Improved |
| add D3S to GRPO | Llama3.1-8B-Instruct · 7 math benchmarks | Pass@8 average: 32.4 -> 40.2 | Improved |
| add D3S to GSPO | Qwen2.5-Math-7B · 7 math benchmarks | Pass@1 average: 33.2 -> 35.8 | Improved |
| add D3S to GSPO | Qwen2.5-Math-1.5B · 7 math benchmarks | Pass@1 average: 33.3 -> 33.3 | No clear effect |
| replace PODS reward-variance selection with D3S | GRPO · 7 math benchmarks (Qwen2.5-Math-7B) | Pass@8 average: 54.4 -> 56.8 | Improved |
| remove cross-group selection (D3S-I) on unaligned Llama (ablation) | D3S · Llama3.1-8B-Instruct, 7 math benchmarks | Pass@1 average: 23.8 -> 24.0 | Improved |
| add cross-group selection (D1S-C vs D1S) on unaligned Llama (ablation) | GRPO · Llama3.1-8B-Instruct, 7 math benchmarks | Pass@8 average: 40.6 -> 33.3 | Hurt |
| add D3S to GRPO on domain-aligned OpenMath2-Llama | GRPO · OpenMath2-Llama3.1-8B | Pass@1 average: 28.2 -> 28.7 | Mixed |
Closest papers by meaning
- Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO (ICML 2026 · similarity 0.90)
- Design from Policies: Conservative Test-Time Adaptation for Offline Policy Optimization (NeurIPS 2023 · similarity 0.89)
- Action-Level Behavior Policy Optimization for Variance-Reduced Policy Gradients (NeurIPS 2026 · similarity 0.88)
- Ranking Policy Gradient (ICLR 2020 · similarity 0.88)
- Reflective Prompted Policy Optimization: Trajectory-Grounded Revision and Salience Bias (NeurIPS 2026 · similarity 0.88)
- Generalized Proximal Policy Optimization with Sample Reuse (NeurIPS 2021 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
