Research map / Large language models / Agents, code and math
JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Distills GPT-4's math problem synthesis ability into a 7B model (DeepSeekMath-7B), uses it with gradient-based data value estimation to synthesize 6M math problems (4.6B tokens) for pre-training JiuZhang3.0 models that reach SOTA among open-source LLMs on math reasoning benchmarks.
Problem
Math reasoning in LLMs typically needs either huge math corpora for pre-training or massive synthetic problems from strong, expensive LLMs such as GPT-4, which makes training and synthesis costly.
Approach
Train a small data-synthesis LLM by distilling GPT-4 on education-stage prompts and gradient-influence-selected math texts, then use that small model to synthesize pre-training data at a fraction of the GPT-4 API cost.
Key results
- JiuZhang3.0-7B reaches 52.8 on MATH vs 50.2 for DeepSeekMath-7B-RL in the natural language reasoning setting.
- JiuZhang3.0-8x7B reaches 89.8 on GSM8k vs 86.4 for MAmmoTH2-8x7B-Plus.
- Total cost is estimated at about 8,480 USD for JiuZhang3.0-7B versus roughly 39,600 to 52,400 USD for KPMath and DeepSeekMath-7B-RL under the paper's own cost estimates.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the 7B trained synthesis model and synthetic pre-training data | JiuZhang3.0-7B · MATH, natural language reasoning | accuracy: 50.2 -> 52.8 | Helped |
| replace the crafted prompt set with a simple prompt (ablation) | JiuZhang3.0 · GSM8K, efficient ablation test | accuracy: 78.6 -> 76.9 | Hurt |
| replace the crafted prompt set with a simple prompt (ablation) | JiuZhang3.0 · MATH, efficient ablation test | accuracy: 32.8 -> 27.8 | Hurt |
| add math-related texts as synthesis inputs (ablation) | JiuZhang3.0 · MATH, efficient ablation test | accuracy: 28.6 -> 32.8 | Helped |
| add boosting retraining of the synthesis model (ablation) | JiuZhang3.0 · MATH, efficient ablation test | accuracy: 31.6 -> 32.8 | Helped |
| use value estimation rather than random sampling for data selection (ablation) | JiuZhang3.0 · MATH, efficient ablation test | accuracy: 31.0 -> 32.8 | Helped |
| regenerate selected high-value instances with GPT-4 for boosting (ablation) | JiuZhang3.0 · MATH, efficient ablation test | accuracy: 28.4 -> 32.8 | Helped |
| replace the trained synthesis model with ChatGPT (ablation) | JiuZhang3.0 · MATH, efficient ablation test | accuracy: 32.8 -> 26.6 | Hurt |
| replace the trained synthesis model with Mixtral-8×7B (ablation) | JiuZhang3.0 · MATH, efficient ablation test | accuracy: 32.8 -> 26.8 | Hurt |
| replace the trained synthesis model with DeepSeekMath-RL-7B (ablation) | JiuZhang3.0 · MATH, efficient ablation test | accuracy: 32.8 -> 27.2 | Hurt |
| replace the trained synthesis model with LLaMA-3-8B-Instruct (ablation) | JiuZhang3.0 · MATH, efficient ablation test | accuracy: 32.8 -> 26.2 | Hurt |
| replace gradient-based selection with random sampling (ablation) | JiuZhang3.0 · MATH, efficient ablation test | accuracy: 32.8 -> 31.0 | Hurt |
Surprising findings
- Existing LLMs such as ChatGPT, Mixtral-8x7B, and LLaMA-3-8B-Instruct used directly as data synthesizers yield worse downstream performance than the distilled small synthesis model.
Limitations
- Synthesis model trained only on math data, so it may be weaker for other domains; MMLU-STEM performance is weak.
- The 4.6B pre-training synthetic data is not quality-filtered and remains large scale.
- Cost estimates for baselines are based on the authors' estimates rather than re-implementation.
Experimental setup
Datasets: OpenWebText Mathpile-textbook Mathpile-arXiv MMIQC StackExchange Mathpile-Wikipedia GSM8k MATH MetaMATH MMIQC Math-Shepherd Orca-MATH PRM800k
Benchmarks: GSM8k MATH SVAMP ASDiv MAWPS CARP TabMWP AQuA SAT-Math MMLU-STEM OCW-Math GSM-Hard
Metrics: accuracy
Baselines: GPT-4 ChatGPT DeepSeekMath-7B-RL MAmmoTH2-8x7B-Plus KPMath-DSMath-7B Rho-1-Math-7B OpenMath-Mistral-7B ToRA-7B-Code
Builds on: DeepSeekMath LoRA MAmmoTH2 Rho-1 Mistral-7B
Closest papers by meaning
- Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages (ICLR 2025 · similarity 0.92)
- OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data (ICLR 2025 · similarity 0.91)
- MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning (ICLR 2024 · similarity 0.90)
- MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code (ICLR 2025 · similarity 0.90)
- Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning (NeurIPS 2025 · similarity 0.89)
- MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.