Research map / Large language models / Agents, code and math
CodePlan: Unlocking Reasoning Potential in Large Language Models by Scaling Code-form Planning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CodePlan, which trains LLMs to generate pseudocode plans before responses, using a 2M-example dataset of (prompt, code plan, response) triples auto-annotated from WebInstruct.
Problem
LLMs have weak multi-step planning, and prior fixes rely on prompting or task-specific fine-tuning that generalize poorly; natural-language pretraining corpora do not expose latent reasoning structure.
Approach
Decompose generation into planning (generate a Python-style pseudocode plan) and surface realization (write the response conditioned on the plan), using explicit code plans annotated by an LLM from existing prompt-response corpora, trained as one shared-parameter LM with a combined loss.
Key results
- Averaged across 13 benchmarks, CodePlan gives a 25.1% relative improvement over vanilla training.
- On Mistral-7B, CodePlan raises GSM8K from 54.1 (vanilla) to 59.5 and Last Letter from 37.5 to 57.5.
- On MuSiQue, relative EM gains grow from 6.3% on 2-hop questions to 43.8% on 4-hop questions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Generate and follow code-form plans instead of directly generating responses | Mistral-7B · GSM8K | accuracy: 54.1 -> 59.5 | Helped |
| Generate and follow code-form plans instead of directly generating responses | Mistral-7B · Last Letter | accuracy: 37.5 -> 57.5 | Helped |
| Generate and follow code-form plans instead of directly generating responses | Mistral-7B · ALFWorld | accuracy: 14.1 -> 23.2 | Mixed |
| Use Llama-3-70B-Instruct instead of Llama-3-8B-Instruct to annotate plans (ablation) | CodePlan · Mathematical Reasoning | average score: 51.7 -> 50.8 | No clear effect |
| Use Gemma-2B-Instruct instead of Llama-3-8B-Instruct to annotate plans (ablation) | CodePlan · Mathematical Reasoning | average score: 51.7 -> 45.1 | Hurt |
| Replace code-form plans with natural-language plans (ablation) | CodePlan · Symbolic reasoning | average score: +27.2% | Helped |
| Replace code-form plans with natural-language plans (ablation) | CodePlan · Decision-making | average score: +27.5% | Helped |
| Replace code-form plans with executable code solutions (ablation) | CodePlan · SVAMP | accuracy | Hurt |
| Replace code-form plans with executable code solutions (ablation) | CodePlan · MATH | accuracy | Helped |
| Filter training examples using plan information gain (ablation) | CodePlan · Downstream reasoning benchmarks | performance | No clear effect |
| Add code-form plans instead of training on prompt-response pairs alone | CodePlan · Response generation NLL | negative log-likelihood: 0.689 -> 0.580 | Helped |
Surprising findings
- Vanilla training on the raw corpus often degrades Mistral-7B by 2 to 9 points on several symbolic, multi-hop QA and decision-making tasks.
- Executable code reasoning (CodeReason) lags vanilla training on MATH and MuSiQue despite strong SVAMP results.
Limitations
- Plans are not executable, so the approach does not use code execution for verification.
- Evaluation of symbolic tasks required fine-tuning each model on 5K synthetic examples to avoid degeneration, confounding comparisons.
- Plan annotations depend on a teacher LLM; plan quality issues can cause subtask errors as shown in the case study.
Experimental setup
Datasets: WebInstruct UltraChat GSM8K MATH SVAMP BBH Boolean Expression BBH Dyck Language Coin Flipping Last Letter Concatenation AlpacaEval MT-Bench MuSiQue
Benchmarks: GSM8K MATH SVAMP BBH Boolean Expression Coin Flipping Last Letter Concatenation BBH Dyck Language AlpacaEval 1.0 AlpacaEval 2.0 MT-Bench MuSiQue HotpotQA
Metrics: accuracy win rate GPT-4 score EM F1
Baselines: Plan-and-Solve prompting Quiet-STaR Vanilla training CodeReason (executable code reasoning) natural language planning
Builds on: chain-of-thought Plan-and-Solve prompting Quiet-STaR WebInstruct
Closest papers by meaning
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.91)
- CodeIO: Condensing Reasoning Patterns via Code Input-Output Prediction (ICML 2025 · similarity 0.90)
- Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignment (ICML 2025 · similarity 0.90)
- Scaling Code-Assisted Chain-of-Thoughts and Instructions for Model Reasoning (NeurIPS 2025 · similarity 0.90)
- Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning (ICML 2025 · similarity 0.90)
- Unveiling the Magic of Code Reasoning through Hypothesis Decomposition and Amendment (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.