Research map / Large language models / Agents, code and math
Learning to Plan Before Answering: Self-Teaching LLMs to Learn Abstract Plans for Problem Solving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LEPA, a self-training method where the LLM generates and self-reflectively refines abstract anticipatory plans before solutions, then is SFT-trained on plan-solution pairs, outperforming ReST, ReST-EM and STaR on MATH, Hellaswag, BoolQ and PIQA.
Problem
Self-training methods for LLMs generate only step-by-step solutions, which fail to capture abstract meta-knowledge needed for generalization across similar problems, limiting performance on hard reasoning tasks like MATH.
Approach
LEPA has the LLM first produce an anticipatory plan (abstract, problem-agnostic) before solving; incorrect plans are refined via self-reflection (without access to the answer in the plan), and the model is SFT-trained on both plan and solution in a two-round conversation.
Key results
- LEPA reaches 30.2% on Hendrycks MATH vs 28.2% for the best baseline ReST, and 73.9% average across four benchmarks vs 70.8% for ReST-EM.
- On Hellaswag LEPA gets 91.2% vs 86.4% for ReST-EM (+4.8%).
- Ablation: removing anticipatory plans drops MATH accuracy to 24.3%, and removing self-reflection to 28.8%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add anticipatory plans before answering | initial Llama 3 8B Instruct with Plan+CoT instead of zero-shot CoT · Hellaswag | accuracy: 60.8% -> 56.1% | Hurt |
| add anticipatory plans before answering | initial Llama 3 8B Instruct with Plan+CoT instead of zero-shot CoT · Hendrycks MATH | accuracy: 19.5% -> 22.1% | Helped |
| add anticipatory plans before answering | initial Llama 3 8B Instruct with Plan+CoT instead of zero-shot CoT · BoolQ | accuracy: 77.3% -> 80.8% | Helped |
| add anticipatory plans before answering | initial Llama 3 8B Instruct with Plan+CoT instead of zero-shot CoT · PIQA | accuracy: 67.0% -> 75.7% | Helped |
| add anticipatory plans to self-training (ablation) | LEPA · Hendrycks MATH | accuracy: 24.3% -> 30.2% | Helped |
| add anticipatory plans to self-training (ablation) | LEPA · BoolQ | accuracy: 84.8% -> 88.4% | Helped |
| add anticipatory plans to self-training (ablation) | LEPA · PIQA | accuracy: 84.5% -> 85.9% | Helped |
| use plan self-reflection instead of rejection sampling (ablation) | LEPA · Hendrycks MATH | accuracy: 28.8% -> 30.2% | Helped |
| use plan self-reflection instead of rejection sampling (ablation) | LEPA · BoolQ | accuracy: 86.9% -> 88.4% | Helped |
| use plan self-reflection instead of rejection sampling (ablation) | LEPA · PIQA | accuracy: 84.8% -> 85.9% | Helped |
| add silence tokens in the solution (ablation) | ReST-based inference-compute variant · Hendrycks MATH | accuracy: 28.2% -> 28.3% | No clear effect |
| train the model to output a new solution after an incorrect initial solution (ablation) | ReST-based inference-compute variant · Hendrycks MATH | accuracy: 28.2% -> 27.8% | Hurt |
Surprising findings
- Plan+CoT prompting of the initial model hurts Hellaswag (56.1% vs 60.8% for CoT), attributed to poor plan calibration.
- STaR's accuracy drops sharply on MATH due to false-positive solutions with wrong rationales but correct final answers.
Limitations
- Experiments limited to 8B Llama models; authors note larger models are unexplored.
- Gains on MATH are modest (about 2 points over the best baseline) and single-run variance is not reported.
- Self-reflection and planning increase inference token cost (about 826 tokens vs 175 for STaR).
Experimental setup
Datasets: Hendrycks MATH Hellaswag BoolQ PIQA CSQA MMLU MMLU-Pro
Benchmarks: Hendrycks MATH Hellaswag BoolQ PIQA MMLU-Pro (Math) CSQA MMLU
Metrics: test accuracy
Baselines: ReST ReST-EM STaR zero-shot CoT
Builds on: STaR ReST ReST-EM Plan-and-Solve prompting zero-shot chain-of-thought
Closest papers by meaning
- Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing (NeurIPS 2024 · similarity 0.88)
- StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving (NeurIPS 2024 · similarity 0.88)
- Non-myopic Generation of Language Models for Reasoning and Planning (ICLR 2025 · similarity 0.88)
- Planning in Natural Language Improves LLM Search for Code Generation (ICLR 2025 · similarity 0.88)
- Planning without Search: Refining Frontier LLMs with Offline Goal-Conditioned RL (NeurIPS 2025 · similarity 0.87)
- ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvement (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.