Research map / Large language models / Instruction and fine-tuning
OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds OpenMathInstruct-1, a permissively licensed 1.8M-pair math instruction dataset by synthesizing code-interpreter solutions for GSM8K and MATH with Mixtral and masked-text-solution prompting, then fine-tunes CodeLlama/Mistral/Llama-2 models on it.
Problem
Math instruction-tuning datasets such as MetaMathQA and MAmmoTH rely on closed-source GPT models with restrictive licenses, and open-source LLMs have lower training-set coverage of correct solutions than GPT-4.
Approach
Use reference text solutions converted into masked text solutions as few-shot prompts (preventing shortcut answers) to raise coverage of correct code-interpreter solutions from an open model (Mixtral), plus fair round-robin downsampling and code-preferred selection.
Key results
- OpenMath-CodeLlama-70B achieves 84.6% on GSM8K and 50.7% on MATH with greedy decoding, with 90.8% and 60.4% under self-consistency (k=50).
- Masked text solution prompting raises MATH training set coverage from 80.1% to 85.9% (default prompt) and GSM8K coverage to 99.9%, with the combined dataset reaching 93% on MATH.
- Fair downsampling beats naive downsampling on the validation split (MATH 37.0 vs 35.0; GSM8K 75.3 vs 74.3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use fair rather than naive downsampling (ablation) | OpenMath fine-tuning data · GSM8K validation subset | accuracy: 74.3 -> 75.3 | Helped |
| use fair rather than naive downsampling (ablation) | OpenMath fine-tuning data · MATH validation subset | accuracy: 35.0 -> 37.0 | Helped |
| use masked-text prompting rather than default prompting (ablation) | OpenMath data synthesis · GSM8K validation subset | accuracy: 73.8 -> 77.7 | Helped |
| use masked-text prompting rather than default prompting (ablation) | OpenMath data synthesis · MATH validation subset | accuracy: 36.9 -> 37.4 | Helped |
| increase fine-tuning dataset size from 128K to 512K (ablation) | Mistral-7B OpenMath model · GSM8K validation subset | accuracy: 75.3 -> 81.0 | Helped |
| increase fine-tuning dataset size from 128K to 512K (ablation) | Mistral-7B OpenMath model · MATH validation subset | accuracy: 37.0 -> 41.6 | Helped |
| use subject-specific rather than default prompts (ablation) | OpenMath MATH data synthesis · MATH training set coverage | coverage: 80.1 -> 80.1 | No clear effect |
| use subject-specific rather than default prompts (ablation) | OpenMath MATH fine-tuning data · MATH validation subset, greedy decoding | accuracy | Hurt |
| use subject-specific rather than default prompts (ablation) | OpenMath MATH fine-tuning data · MATH validation subset, self-consistency with four samples | accuracy | Helped |
| use code-preferential solution selection (ablation) | OpenMath MATH fine-tuning data · MATH validation subset, greedy decoding | accuracy | Helped |
| use code-preferential solution selection (ablation) | OpenMath MATH fine-tuning data · MATH validation subset, self-consistency decoding | accuracy | Hurt |
| use masked-text rather than unmasked reference solutions in prompts (ablation) | Mixtral solution synthesis · MATH training set coverage | coverage: 80.1% -> 85.9% | Helped |
Surprising findings
- Naive text-reference-solution prompting produced many shortcut solutions (e.g., print(ANSWER)), which motivated masking intermediate computations.
Limitations
- Semantically noisy solutions (correct answer via flawed reasoning) are not filtered.
- Gains on in-domain benchmarks may not transfer; performance drops on GSM-Hard suggest limited robustness to perturbations.
- Only GSM8K and MATH training problems are used; no new problems are synthesized.
Experimental setup
Datasets: GSM8K MATH
Benchmarks: GSM8K MATH GSM-Hard SVAMP TabMWP ASDiv MAWPS
Metrics: accuracy training set coverage (pass@k)
Baselines: GPT-4 (Code Interpreter) WizardMath MetaMath MAmmoTH ToRA
Builds on: MetaMathQA MAmmoTH ToRA Chain-of-thought
Closest papers by meaning
- OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data (ICLR 2025 · similarity 0.94)
- MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning (ICLR 2024 · similarity 0.91)
- MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning (ICLR 2024 · similarity 0.89)
- MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models (ICLR 2024 · similarity 0.89)
- MathScale: Scaling Instruction Tuning for Mathematical Reasoning (ICML 2024 · similarity 0.89)
- Multilingual Mathematical Reasoning: Advancing Open-Source LLMs in Hindi and English (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.