Research map / Reinforcement learning / RL from feedback
The Easy, the Hard, and the Learnable: Confidence and Difficulty-Adaptive Policy Optimization for LLM Reasoning
Zhanke Zhou, Xiangyu Lu, Chentao Cao, Brando Miranda, Tongliang Liu, Bo Han, Sanmi Koyejo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CoDaPO (CoDaWeighting, CoDaSampling, CoDaLearning) to GRPO | GRPO, Qwen2.5-Math-1.5B · 7 math benchmarks average | average accuracy (%): 39.08 -> 41.30 | Improved |
| add CoDaPO to GRPO on Qwen2.5-Math-7B | GRPO, Qwen2.5-Math-7B · 7 math benchmarks average | average accuracy (%): 44.58 -> 46.67 | Improved |
| add CoDaPO to GRPO on Llama-3.2-1B-Instruct | GRPO, Llama-3.2-1B-Instruct · 7 math benchmarks average | average accuracy (%): 13.48 -> 15.05 | Improved |
| add CoDaWeighting to GRPO (ablation) | GRPO, Qwen2.5-Math-1.5B · MATH500/AIME24/AIME25 average | average accuracy (%): 30.44 -> 31.53 | Improved |
| add CoDaSampling on top of CoDaWeighting (ablation) | CoDaPO, Qwen2.5-Math-1.5B · MATH500/AIME24/AIME25 average | average accuracy (%): 31.53 -> 32.79 | Improved |
| add confidence term Vc(c)=c to difficulty-only weighting (ablation) | CoDaWeighting · MATH500/AIME24/AIME25 average | average accuracy (%): 31.32 -> 31.53 | Improved |
| add difficulty term Vd to confidence-only weighting (ablation) | CoDaWeighting · MATH500/AIME24/AIME25 average | average accuracy (%): 30.99 -> 31.53 | Improved |
| use Top-K=4 instead of Top-K=1 in CoDaSampling (ablation) | CoDaPO · MATH500/AIME24/AIME25 average | average accuracy (%): 30.40 -> 32.79 | Improved |
| vary Top-K among 2, 4, 8 (ablation) | CoDaSampling · MATH500/AIME24/AIME25 average | average accuracy (%): 32.48 / 32.79 / 32.62 | No clear effect |
| apply CoDaLearning to DAPO | DAPO · 7 math benchmarks average | average accuracy (%): 39.49 -> 41.03 | Improved |
| apply CoDaLearning to GPG | GPG · 7 math benchmarks average | average accuracy (%): 39.77 -> 39.95 | Improved |
| use easy-to-hard curriculum instead of uniform sampling | GRPO · 7 math benchmarks average | average accuracy (%): 39.08 -> 39.30 | No clear effect |
Closest papers by meaning
- Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning (ICLR 2026 · similarity 0.91)
- Nudging the Boundaries of LLM Reasoning (ICLR 2026 · similarity 0.90)
- Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts (NeurIPS 2025 · similarity 0.90)
- One-Way Policy Optimization for Self-Evolving LLMs (ICML 2026 · similarity 0.90)
- Advancing LLM Reasoning with Natural Language and Numerical Feedback (ICML 2026 · similarity 0.89)
- General-Reasoner: Advancing LLM Reasoning Across All Domains (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
