Research map / Large language models / Reasoning and chain of thought

MathFimer: Enhancing Mathematical Reasoning by Expanding Reasoning Steps through Fill-in-the-Middle Task

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains a fill-in-the-middle model (MathFimer-7B) on NuminaMath-FIM to insert intermediate steps into existing math CoT solutions, and shows SFT on the expanded data improves GSM8K, MATH, Math Odyssey and OlympiadBench-EN accuracy.

Problem

Existing math reasoning training data often has coarse-grained solution steps, and current step-expansion methods rely on stronger external models or expensive search (e.g., MCTS), which can introduce errors by generating new chains instead of building on verified solutions.

Approach

Adapt the code fill-in-the-middle paradigm to math solutions: decompose verified CoT solutions into steps, hold out one middle step with prefix/suffix context, and train a 7B model (from Qwen2.5-Math-7B) to reconstruct it, then insert generated steps between existing steps (filtered by a similarity threshold) to create expanded datasets.

Key results

• MathFimer-expanded MathInstruct-CoT raises Meta-Llama-3.1-8B average accuracy from 27.75% to 31.52% across the four math benchmarks.
• On Qwen2.5-Math-72B, MetaMathQA expansion raises average accuracy from 52.71% to 57.14%.
• Iterative expansion on MI-CoT with Llama-3.1-8B raises GSM8K accuracy from 67.78% to 83.32% over three iterations (own table, MI-CoT base).

Surprising findings

• MathFimer-1.5B, 7B and 72B give nearly identical downstream results, suggesting step expansion is not bottlenecked by model capacity.
• Expansion on Qwen2.5-Math-7B with MetaMathQA slightly regresses GSM8K (93.18 to 93.10) and some settings show drops, so gains are not uniform.

Limitations

• Generated steps may contain errors that are hard to control, causing occasional regressions; no robust verification of inserted steps.
• Less suitable for already-detailed traces (e.g., R1-like) or non-linear, tree-shaped reasoning, and code reasoning where insertion breaks executability.
• Gains from expansion are partly confounded with distillation effects, and the expanded data inherits base-model biases.

Experimental setup

Datasets: NuminaMath-CoT GSM8K MATH MathInstruct-CoT MetaMathQA ScaleQuest-Math DAPO-Math-17K

Benchmarks: GSM8K MATH Math Odyssey OlympiadBench-EN BBH GPQA_diamond MMLU_redux LogicBench HellaSwag

Metrics: average accuracy over 16 samples at temperature 0.7 PRM score

Baselines: prompt-based step expansion (Llama-3.2-3B-Instruct) DPO Step-DPO RFT PPO

Builds on: fill-in-the-middle training chain-of-thought prompting Qwen2.5-Math NuminaMath-CoT

Closest papers by meaning

• Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages (ICLR 2025 · similarity 0.91)
• MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code (ICLR 2025 · similarity 0.91)
• MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning (ICLR 2024 · similarity 0.91)
• UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models (ICLR 2025 · similarity 0.90)
• DenseSteer: Steering Small Language Models towards Dense Math Reasoning (ICML 2026 · similarity 0.90)
• MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy (AAAI 2026 · similarity 0.90)