Research map / Large language models / Reasoning and chain of thought
SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SELF-DISCOVER, a two-stage zero-shot framework where an LLM selects, adapts, and implements atomic reasoning modules into a task-level JSON-like reasoning structure that it then follows at inference.
Problem
Typical prompting methods (CoT, Plan-and-Solve, etc.) impose a single fixed reasoning process, which is suboptimal for diverse complex reasoning tasks; inference-heavy ensembles like self-consistency are costly.
Approach
LLMs run a task-level meta-reasoning stage (SELECT, ADAPT, IMPLEMENT) over a seed set of 39 atomic reasoning modules to compose an explicit reasoning structure, which is then appended to each instance, requiring only about 3 extra task-level calls plus one call per instance.
Key results
- On GPT-4, Self-Discover reaches 81% on BBH, 85% on T4D and 73% on MATH, versus 75%, 52% and 71% for CoT.
- On PaLM 2-L, Self-Discover reaches 67% BBH, 69% T4D and 50.5% MATH, versus 60%, 40% and 42% for CoT.
- Self-Discover outperforms CoT-self-consistency and majority voting of each RM while using 10-40x fewer inference calls per instance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Self-Discover instead of CoT | PaLM 2-L · BBH (23-task aggregate) | accuracy: +7% absolute improvement | Helped |
| Use Self-Discover instead of Plan-and-Solve | PaLM 2-L · BBH (23-task aggregate) | accuracy: +6% absolute improvement | Helped |
| Use Self-Discover instead of CoT | GPT-4 · BBH (23-task aggregate) | accuracy: +6% | Helped |
| Use Self-Discover instead of Plan-and-Solve | GPT-4 · BBH (23-task aggregate) | accuracy: +8% | Helped |
| Use Self-Discover instead of the baselines | PaLM 2-L · T4D | accuracy: ≥27% | Helped |
| Use Self-Discover instead of the baselines | GPT-4 · T4D | accuracy: 32% | Helped |
| Use Self-Discover instead of baselines | PaLM 2-L · MATH | accuracy: 1%-7% | Helped |
| Use Self-Discover instead of baselines | GPT-4 · MATH | accuracy: 2%-3% | Helped |
| Add ADAPT to SELECT (ablation) | Self-Discover · 4 reasoning tasks | zero-shot reasoning performance | Helped |
| Add IMPLEMENT to SELECT and ADAPT (ablation) | Self-Discover · 4 reasoning tasks | zero-shot reasoning performance | Helped |
| Apply PaLM 2-L-discovered structures instead of optimized OPRO prompts | GPT-4 · 4 reasoning tasks | accuracy | Mixed |
| Apply GPT-4-discovered structures instead of CoT | Llama2-70B · BBH disambiguation QA | accuracy: 52% vs 42% | Helped |
Surprising findings
- Structures discovered by PaLM 2-L transfer to GPT-4, and structures discovered by GPT-4 transfer to Llama2-70B, improving over CoT.
- On MATH error analysis, 87.5% of self-discovered reasoning structures were correct, and 74.7% of failures came from computation errors rather than structure errors.
Limitations
- Smaller open models such as Llama2 struggle to generate good reasoning structures by themselves, so structures must come from a stronger model.
- Structure is longer than CoT prompts, increasing per-instance input and output cost.
- Computation errors (e.g., arithmetic) remain the dominant failure mode on MATH.
Experimental setup
Datasets: BIG-Bench Hard Thinking for Doing (T4D) MATH
Benchmarks: BIG-Bench Hard Thinking for Doing (T4D) MATH
Metrics: accuracy exact match number of inference calls
Baselines: Direct prompting CoT Plan-and-Solve CoT-Self-Consistency Majority voting of each RM Best of each RM OPRO Foresee and Reflect (FaR)
Builds on: chain-of-thought prompting plan-and-solve prompting least-to-most prompting OPRO self-consistency
Closest papers by meaning
- ReGenesis: LLMs can Grow into Reasoning Generalists via Self-Improvement (ICLR 2025 · similarity 0.90)
- On the self-verification limitations of large language models on reasoning and planning tasks (ICLR 2025 · similarity 0.89)
- Scalable Best-of-N Selection for Large Language Models via Self-Certainty (NeurIPS 2025 · similarity 0.89)
- Chain-of-Thought Reasoning Without Prompting (NeurIPS 2024 · similarity 0.89)
- Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing (NeurIPS 2024 · similarity 0.88)
- Incentivizing LLMs to Self-Verify Their Answers (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.