Research map / Large language models / Reasoning and chain of thought

SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correction

ICLR 2025 · Large language models › Reasoning and chain of thought · approach: Routing, pruning and distillation for reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SuperCorrect, a two-stage method that distills hierarchical thought templates from a large teacher (o1-mini/GPT-4o-mini) into small student LLMs via SFT, then applies cross-model collaborative DPO on teacher-corrected error steps to improve self-correction, reaching SOTA among 7B models on MATH and GSM8K.

Problem

Smaller LLMs struggle with complex mathematical reasoning and cannot reliably locate and correct errors in their own reasoning steps, and self-correction without accurate error localization is ineffective.

Approach

Use a hierarchical thought template (detailed key explanations plus a generalized high-level summary) distilled from a strong teacher for SFT, then train the student with DPO where the preferred option is the teacher's error analysis and correction of the first wrong step and the rejected option is the student's own analysis of the same step.

Key results

• SuperCorrect-Qwen-7B reaches 70.2% on MATH and 89.5% on GSM8K, versus 55.1%/83.2% for Qwen2.5-Math-7B-Instruct.
• SuperCorrect-DeepSeek-7B reaches 54.6% MATH and 88.2% GSM8K, versus 46.8%/82.9% for DeepSeekMath-7B-Instruct.
• On Qwen2.5-Math, cross-model DPO adds +7.1 on MATH over the HSFT model (62.4 to 70.2) and beats Reflexion (63.1).

Measured effects reported in the paper

Surprising findings

• Reflexion-style self-correction on the base model gave limited or negative gains for most LLMs, while the cross-model DPO gave about 5-6% gains in self-correction accuracy.
• Standard SFT on the same data gave only marginal gains (55.1 to 57.4 on MATH for Qwen2.5-Math), well below HSFT.

Limitations

• Relies on closed-source frontier teachers (o1-mini, GPT-4o) to generate thought templates and corrections, adding cost and dependence.
• Evaluated only on 7-8B student models; the authors plan to extend to larger models.
• Paper contains many unresolved [ref]/[cite] placeholders and inconsistent numbers across tables, making some claims hard to verify.

Experimental setup

Datasets: MATH training set GSM8K training set GaoKao Bench (670 translated problems) NuminaMath MetaMath

Benchmarks: MATH GSM8K

Metrics: accuracy locate correctness correction accuracy mean and variance of accuracy over repeated runs

Baselines: Step-DPO Reflexion Qwen2.5-Math-7B-Instruct DeepSeekMath-7B-Instruct Meta-Llama3.1-8B-Instruct GPT-4o Llama-3-70B-Instruct MetaMath MAmmoTH2-7B-Plus Xwin-Math-Mistral-7B

Builds on: DPO Step-DPO Reflexion Chain-of-Thought Buffer of Thoughts

Closest papers by meaning

• S^3cMath: Spontaneous Step-Level Self-Correction Makes Large Language Models Better Mathematical Reasoners (AAAI 2025 · similarity 0.90)
• UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Models (ICLR 2025 · similarity 0.90)
• Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages (ICLR 2025 · similarity 0.89)
• Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems (ICLR 2025 · similarity 0.89)
• Reasoning Scaffolding: Distilling the Flow of Thought from LLMs (ICLR 2026 · similarity 0.89)
• Turning Dust into Gold: Distilling Complex Reasoning Capabilities from LLMs by Leveraging Negative Data (AAAI 2024 · similarity 0.89)