Research map / Large language models / Reasoning and chain of thought
Measuring Mathematical Problem Solving With the MATH Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pretrain on AMPS (ablation) | GPT-2 0.1B vs GPT-3 13B without AMPS · MATH | accuracy: 5.2% (13B no AMPS) vs 5.4% (0.1B AMPS) | Helped |
| add Math StackExchange to AMPS pretraining (ablation) | GPT-2 0.3B · MATH | accuracy: 6.2 -> 6.0 | Hurt |
| scale GPT-2 from 0.1B to 1.5B | GPT-2 · MATH | accuracy: 5.4 -> 6.9 | Helped |
| generate step-by-step solution at test time (scratch space) (ablation) | GPT-2 1.5B · MATH | accuracy: 6.9 -> 5.3 | Hurt |
| train on step-by-step solutions (ablation) | GPT-2 1.5B · MATH | accuracy: 6.3 -> 6.9 | Helped |
| give partial ground-truth solution as hint | GPT-2 0.7B · MATH | accuracy | Helped |
| use BART encoder-decoder instead of GPT-2 | AMPS-pretrained model · MATH | accuracy: 4.9% | No clear effect |
Closest papers by meaning
- MathScale: Scaling Instruction Tuning for Mathematical Reasoning (ICML 2024 · similarity 0.89)
- Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages (ICLR 2025 · similarity 0.88)
- DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving (NeurIPS 2024 · similarity 0.88)
- Machine Learning meets Algebraic Combinatorics: A Suite of Datasets Capturing Research-level Conjecturing Ability in Pure Mathematics (ICML 2025 · similarity 0.87)
- MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code (ICLR 2025 · similarity 0.87)
- OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalization (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.