Research map / Large language models / Instruction and fine-tuning
DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Difficulty-Aware Rejection Tuning (DART-Math), which allocates more sampling trials to hard math queries using a 7B open model (DeepSeekMath-7B-RL), yielding smaller datasets (DART-Math-Uniform and DART-Math-Hard, ~590k each) that outperform vanilla rejection tuning and prior public math SFT datasets.
Problem
Rejection-sampling-based synthetic math instruction data is biased toward easy queries, with many of the hardest training queries receiving zero correct responses, which hampers learning of complex mathematical reasoning.
Approach
Instead of sampling a fixed number of responses per query, sample until each query reaches a target number of correct responses (Uniform) or a difficulty-proportional number (Prop2Diff), with difficulty measured by the fail rate of a 7B open-weight model; this explicitly controls the final training distribution and covers hard queries.
Key results
- DART-Math-Llama3-8B (Prop2Diff) improves MATH from 39.7% (VRT) to 46.6% and reaches 39.7 average over six benchmarks with 0.59M samples.
- DART-Math-Mistral-7B (Prop2Diff) averages 38.8 versus 34.3 for VRT and surpasses Mistral-7B-MMIQC (34.2) with about a quarter of its training samples.
- Gains over VRT narrow on stronger base models: DART-Math-DSMath-7B reaches 49.4 average vs 48.3 for VRT, with the improvement mostly on OOD benchmarks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use difficulty-aware Uniform sampling instead of vanilla rejection tuning | Llama3-8B · MATH | accuracy: 39.7 -> 45.3 | Helped |
| Use difficulty-aware Prop2Diff sampling instead of vanilla rejection tuning | Llama3-8B · MATH | accuracy: 39.7 -> 46.6 | Helped |
| Use difficulty-aware Prop2Diff sampling instead of vanilla rejection tuning | Llama3-8B · GSM8K | accuracy: 81.7 -> 81.1 | Hurt |
| Use difficulty-aware Prop2Diff sampling instead of vanilla rejection tuning | Llama3-8B · TheoremQA | accuracy: 14.9 -> 19.4 | Helped |
| Use difficulty-aware Prop2Diff sampling instead of vanilla rejection tuning | Mistral-7B · MATH | accuracy: 38.7 -> 45.5 | Helped |
| Use difficulty-aware Prop2Diff sampling instead of vanilla rejection tuning | Mistral-7B · GSM8K | accuracy: 82.3 -> 81.1 | Hurt |
| Use difficulty-aware Prop2Diff sampling instead of vanilla rejection tuning | Mistral-7B · DeepMind-Mathematics | accuracy: 35.6 -> 45.1 | Helped |
| Enforce at least one synthetic correct response per query in Prop2Diff (ablation) | DART-Math Prop2Diff · GSM8K | accuracy: about 8 points | Helped |
| Enforce at least one synthetic response per query in vanilla rejection tuning (ablation) | Vanilla rejection tuning · MATH | accuracy | No clear effect |
Surprising findings
- Vanilla rejection sampling with DeepSeekMath-7B-RL yields zero correct responses for 51.1% of the hardest MetaMathQA-MATH queries, despite pass@k above 99% with enough samples.
- For DeepSeekMath-7B, SFT on DART-Math performs comparably to its own RL-trained counterpart (49.2-49.4 vs 49.3 average).
Limitations
- Fail rate is a proxy for difficulty and may be sub-optimal compared with other difficulty metrics.
- Limited to natural language reasoning without code execution.
- Gains over vanilla rejection tuning are small on strong math-pretrained base models such as DeepSeekMath-7B and Llama3-70B.
Experimental setup
Datasets: MATH training set GSM8K training set MetaMathQA-MATH-AnsAug MMIQC MetaMath
Benchmarks: MATH GSM8K CollegeMath DeepMind-Mathematics OlympiadBench-Math TheoremQA
Metrics: accuracy average accuracy
Baselines: MetaMath MMIQC KPMath-Plus Xwin-Math WizardMath Vanilla rejection tuning (VRT) DeepSeekMath-7B-Instruct ToRA-style re-implementation MARIO-style re-implementation
Builds on: rejection tuning MetaMath MMIQC vLLM DeepSeekMath
Closest papers by meaning
- MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbations (ICML 2025 · similarity 0.89)
- MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy (AAAI 2026 · similarity 0.89)
- UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models (ICLR 2025 · similarity 0.88)
- HARDMath: A Benchmark Dataset for Challenging Problems in Applied Mathematics (ICLR 2025 · similarity 0.88)
- MARGE: Improving Math Reasoning with Guided Exploration (ICML 2025 · similarity 0.88)
- Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.