Research map / Large language models / Agents, code and math
Key-Point-Driven Data Synthesis with Its Enhancement on Mathematical Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Key-Point-Driven Data Synthesis (KPDDS), which mines key points and topics from seed math problems and synthesizes new QA pairs with GPT-4 quality scoring and voting, producing KPMath/KPMath-Plus datasets used to fine-tune models that reach 87.0% GSM8K and 58.3% MATH with Qwen1.5-72B.
Problem
LLMs' mathematical reasoning is limited by the scarcity of high-quality, reasoning-focused training data, and existing synthetic augmentation methods yield questions that are either too similar to seeds or distributionally off from real data.
Approach
Extract topics and key points from authentic seed problems, build a topic co-occurrence probability matrix (TCPM) to sample topic combinations, generate new questions conditioned on key points and exemplar practices, filter them with a GPT-4 quality score (threshold 0.85), and keep answers via multi-sample consensus voting.
Key results
- KPMath-Plus fine-tuned Qwen1.5-72B reaches 87.0% on GSM8K and 58.3% on MATH zero-shot PASS@1, with a six-benchmark average of 81.5%.
- KPMath-Plus-Mistral-7B averages 75.4% across six math benchmarks, exceeding listed 7B baselines such as MetaMath (Mistral) at 70.8%.
- Semi-voting with CSV threshold 0.1 cut KPMath-M data by 46.7% without performance degradation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune Qwen1.5-72B on KPMath-Plus instead of using the vanilla base model | Qwen1.5-72B · GSM8K | PASS@1 accuracy: 77.6 -> 87.0 | Helped |
| Fine-tune Qwen1.5-72B on KPMath-Plus instead of using the vanilla base model | Qwen1.5-72B · MATH | PASS@1 accuracy: 38.2 -> 58.3 | Helped |
| Add KPMath-G to MixMath (ablation) | Mistral-7B · GSM8K | accuracy: 75.7 -> 80.7 | Helped |
| Add KPMath-G to MixMath (ablation) | Mistral-7B · MATH | accuracy: 43.3 -> 43.0 | Hurt |
| Add KPMath-G to MixMath (ablation) | Mistral-7B · SVAMP | accuracy: 73.6 -> 76.7 | Helped |
| Add KPMath-G to MixMath (ablation) | Mistral-7B · TabMWP | accuracy: 63.1 -> 60.0 | Hurt |
| Add KPMath-G to MixMath (ablation) | Mistral-7B · ASDiv | accuracy: 82.9 -> 85.1 | Helped |
| Add KPMath-G to MixMath (ablation) | Mistral-7B · MAWPS | accuracy: 89.1 -> 93.9 | Helped |
| Add KPMath-M to MixMath (ablation) | Mistral-7B · GSM8K | accuracy: 75.7 -> 77.0 | Helped |
| Add KPMath-M to MixMath (ablation) | Mistral-7B · MATH | accuracy: 43.3 -> 45.9 | Helped |
| Add KPMath-M to MixMath (ablation) | Mistral-7B · SVAMP | accuracy: 73.6 -> 74.0 | Helped |
| Add KPMath-M to MixMath (ablation) | Mistral-7B · TabMWP | accuracy: 63.1 -> 65.0 | Helped |
Surprising findings
- Adding KPMath-G to MixMath reduces MATH accuracy slightly (43.3 to 43.0), which the authors attribute to higher complexity of those tasks.
Limitations
- Synthetic question generation depends on GPT-4 for generation, scoring, and labeling, so quality relies on a proprietary model.
- Data contamination check found some n-gram hits with MATH test sets (102 question hits for KPMath), although the authors judged them low risk after manual review.
- Only CoT-style answers are synthesized; program-based answers are not explored.
Experimental setup
Datasets: MATH training set GSM8K training set MetaMath MMIQC Open-Platypus TAL-SCQ5K-EN MathInstruct CoT subset
Benchmarks: GSM8K MATH SVAMP TabMWP ASDiv MAWPS Hungarian Exam
Metrics: zero-shot PASS@1 accuracy average accuracy
Baselines: GPT-4 ChatGPT Claude-2 PaLM-2 Llama-2 SFT MAmmoTH WizardMath Platypus-2 MetaMath MMIQC DeepSeekMath Llemma
Builds on: chain-of-thought MetaMath MMIQC MathInstruct LLaMA-Factory
Closest papers by meaning
- JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Models (NeurIPS 2024 · similarity 0.89)
- GRIP: A Graph-Based Reasoning Instruction Producer (NeurIPS 2025 · similarity 0.89)
- OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data (ICLR 2025 · similarity 0.88)
- Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages (ICLR 2025 · similarity 0.87)
- RMath: A Logic Reasoning-Focused Datasets Toward Mathematical Multistep Reasoning Tasks (AAAI 2025 · similarity 0.87)
- MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task (ICLR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.