Research map / Large language models / Reasoning and chain of thought

MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Synthesizes hard math problems from random PlanetMath concept-explanation pairs using an SFT-then-GRPO Qwen3-8B generator rewarded for format, reasoning-trace length, and answer consistency.

Problem

Scarcity of high-quality, high-difficulty math training problems limits LLM reasoning; existing synthesis methods rely on transforming human-written templates and lack difficulty control.

Approach

Constructs new problems from scratch by sampling PlanetMath concept-explanation pairs, uses nine difficulty strategies as soft constraints, and trains the generator with GRPO using a reward that favors longer teacher reasoning traces (complexity proxy) and majority-vote answer consistency.

Key results

• MathSmith-HC on Qwen3-8B long-CoT reaches 71.8 average on hard benchmarks vs 67.9 for PromptCOT (+9.8% relative over the base-level average reported in the table).
• On Qwen-2.5-7B-Instruct short-CoT, MathSmith-HC reaches 24.4 average on hard benchmarks (+18.1% relative to the base model).
• MathSmith-HC achieves 95.38% available ratio vs 71.50% for MathSmith-SFT in the ablation.

Surprising findings

• MathSmith-HC underperforms the base model on GSM8K under long-CoT (89.2 vs 94.8 for base Qwen2.5-7B-instruct style rows), which the authors attribute to format mismatch.
• The paper itself notes it remains uncertain whether longer-reasoning problems actually improve LLM reasoning.

Limitations

• Reasoning-trace length is only a heuristic proxy for problem difficulty; the authors acknowledge uncertainty whether such problems improve reasoning.
• Gains on GSM8K are sometimes below the base model, attributed to format mismatch with competition-style synthesis.
• Relies on GPT-4o for cold-start data and on a large teacher (Qwen3-30B-A3B) for reward computation.

Experimental setup

Datasets: PlanetMath concept-explanation pairs (~11k) OpenMathInstruct-2 NuminaMath-COT PromptCOT

Benchmarks: GSM8K MATH-500 AIME2024 AIME2025 OlympiadBench

Metrics: Pass@1 average reasoning trace length available ratio

Baselines: OpenMathInstruct-2 NuminaMath-COT MetaMath PromptCOT

Builds on: GRPO PromptCOT verl LlamaFactory

Closest papers by meaning

• MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task (ICLR 2026 · similarity 0.90)
• AbstRaL: Augmenting LLMs' Reasoning by Reinforcing Abstract Thinking (ICLR 2026 · similarity 0.90)
• Verifier-Backed Hard Problem Generation for Mathematical Reasoning (NeurIPS 2026 · similarity 0.90)
• WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct (ICLR 2025 · similarity 0.90)
• UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models (ICLR 2025 · similarity 0.90)
• OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalization (NeurIPS 2025 · similarity 0.90)