Research map / Reinforcement learning / RL from feedback
Harder Is Better: Boosting Mathematical Reasoning via Difficulty-Aware GRPO and Multi-Aspect Question Reformulation
Yanqi Dai, Yuxiang Ji, Xiao Zhang, Yong Wang, Xiangxiang Chu, Zhiwu Lu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add difficulty-balanced advantage estimation (DGAE) (ablation) | DGPO · Math benchmarks avg (6 sets, trained on MATH) | average accuracy: 37.71 -> 38.65 | Improved |
| add difficulty-aware question weighting (DQW) (ablation) | DGPO · Math benchmarks avg (6 sets, trained on MATH) | average accuracy: 38.65 -> 39.79 | Improved |
| temperature T=2.0 instead of 1.0 in DQW (ablation) | DGPO · Math benchmarks avg (6 sets, trained on MATH) | average accuracy: 39.03 -> 39.79 | Improved |
| temperature T=2.0 instead of 10.0 in DQW (ablation) | DGPO · Math benchmarks avg (6 sets, trained on MATH) | average accuracy: 39.27 -> 39.79 | Improved |
| replace GRPO with DGPO | Qwen2.5-Math-7B · Math benchmarks avg (6 sets, trained on MATH) | average accuracy: 37.61 -> 39.79 | Improved |
| add MQR data augmentation to DGPO (ablation) | Qwen2.5-Math-7B · Math benchmarks avg (6 sets, trained on MATH) | average accuracy: 39.90 -> 42.17 | Improved |
| add MQR data to GRPO-trained model setup (MQR alone) | Qwen2.5-Math-7B · Math benchmarks avg (6 sets, trained on MATH) | average accuracy: 37.61 -> 41.04 | Improved |
| add DGPO on top of DAPO (ablation) | DAPO · Math benchmarks avg (DGPO combined with other methods) | average accuracy: 37.94 -> 39.91 | Improved |
| add DGPO on top of GPG (ablation) | GPG · Math benchmarks avg (DGPO combined with other methods) | average accuracy: 37.93 -> 38.92 | Improved |
| add DGPO on top of GSPO (ablation) | GSPO · Math benchmarks avg (DGPO combined with other methods) | average accuracy: 37.71 -> 39.32 | Improved |
| use DGPO instead of GRPO on multimodal GeoQA | Qwen2.5-VL · GeoQA (multimodal) | accuracy: 57.43 -> 59.95 | Improved |
| MQR reformulation instead of MetaMath rephrasing (ablation) | DGPO · Math benchmarks avg (6 sets, trained on MATH) | average accuracy: 40.73 -> 42.17 | Improved |
Closest papers by meaning
- D²Evo: Dual Difficulty-Aware Self-Evolution for Data-Efficient Reinforcement Learning (ICML 2026 · similarity 0.90)
- QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation (ICLR 2026 · similarity 0.89)
- MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy (AAAI 2026 · similarity 0.89)
- Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning (ICLR 2026 · similarity 0.88)
- Think Smarter not Harder: Adaptive Reasoning with Inference Aware Optimization (ICML 2025 · similarity 0.88)
- Rectify Evaluation Preference: Improving LLMs’ Critique on Math Reasoning via Perplexity-aware Reinforcement Learning (AAAI 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
