Research map / Large language models / Language, safety and interpretability
Large Language Models Cannot Self-Correct Reasoning Yet
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that LLMs cannot intrinsically self-correct reasoning without oracle labels, performance often drops, and that multi-agent debate does not beat self-consistency and prompt-design confounds explain claimed gains.
Problem
Prior work reports large gains from LLM self-correction on reasoning, but these gains may rely on oracle labels, unequal inference cost, or sub-optimal initial prompts.
Approach
Defines intrinsic self-correction (no external feedback or oracle labels) and re-evaluates prior self-correction, multi-agent debate, and prompting setups under fair conditions, finding that gains vanish or reverse.
Key results
- On GSM8K with GPT-4-Turbo, intrinsic self-correction drops accuracy from 91.5 to 88.0 after round 1 and 90.0 after round 2; Llama-2 drops from 62.0 to 36.5 on GSM8K after two rounds.
- With GPT-3.5 on CommonSenseQA, intrinsic self-correction drops accuracy from 75.8 to 41.8 after two rounds, while oracle-label self-correction raises it to 89.7.
- On GSM8K, multi-agent debate with 9 responses reaches 83.0 versus 88.2 for self-consistency with 9 responses.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use oracle labels to decide whether to stop self-correction | GPT-3.5 and GPT-4 · GSM8K, CommonSenseQA, and HotpotQA | accuracy / exact match: 75.9 -> 84.3; 75.8 -> 89.7; 26.0 -> 29.0; 95.5 -> 97.5; 82.0 -> 85.5; 49.0 -> 59.0 | Helped |
| Add one round of intrinsic self-correction | GPT-3.5 · GSM8K | accuracy: 75.9 -> 75.1 | Hurt |
| Add two rounds of intrinsic self-correction | GPT-3.5 · CommonSenseQA | accuracy: 75.8 -> 41.8 | Hurt |
| Add one round of intrinsic self-correction | GPT-4 · GSM8K | accuracy: 95.5 -> 91.5 | Hurt |
| Add two rounds of intrinsic self-correction | GPT-4 · HotpotQA | exact match: 49.0 -> 43.0 | Hurt |
| Add one round of intrinsic self-correction | GPT-4-Turbo · GSM8K | accuracy: 91.5 -> 88.0 | Hurt |
| Add two rounds of intrinsic self-correction | Llama-2 · GSM8K | accuracy: 62.0 -> 36.5 | Hurt |
| Use a different feedback prompt: 'Verify whether your answer is correct, and provide an explanation.' (ablation) | GPT-4-Turbo intrinsic self-correction · GSM8K | accuracy: 91.5 -> 91.0; 91.5 -> 91.0 | Mixed |
| Use a different feedback prompt: 'Review your previous answer and determine whether it's correct. If wrong, find the problems with your answer.' (ablation) | GPT-4-Turbo intrinsic self-correction · CommonSenseQA | accuracy: 84.0 -> 74.5; 84.0 -> 81.0 | Hurt |
| Use a different feedback prompt (ablation) | Llama-2 intrinsic self-correction · GSM8K and CommonSenseQA | accuracy: GSM8K: 62.0 -> 58.0; CommonSenseQA: 64.0 -> 24.0 | Mixed |
| Use self-consistency instead of multi-agent debate with the same number of responses | GSM8K · GSM8K | accuracy: 83.2 -> 85.3; 83.0 -> 88.2 | Helped |
| Use multi-agent debate instead of standard prompting | GSM8K · GSM8K | accuracy: 76.7 -> 83.2 | Helped |
Surprising findings
- Self-correction with oracle labels inflates gains, which vanish once oracle labels are removed.
- Multi-agent debate offers little benefit over self-consistency with the same number of responses.
Limitations
- Focuses on reasoning tasks; self-correction for style or safety alignment may behave differently.
- Evaluates a limited set of models (GPT-3.5, GPT-4, GPT-4-Turbo, Llama-2-70b) and small subsets for some models.
Experimental setup
Datasets: GSM8K CommonSenseQA HotpotQA CommonGen-Hard
Benchmarks: GSM8K CommonSenseQA HotpotQA CommonGen-Hard
Metrics: accuracy exact match concept coverage
Baselines: Standard Prompting Self-Consistency Multi-Agent Debate RCI Reflexion Self-Refine
Builds on: Self-Refine Reflexion Multi-Agent Debate self-consistency chain-of-thought
Closest papers by meaning
- Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs (NeurIPS 2025 · similarity 0.94)
- Small Language Model Can Self-Correct (AAAI 2024 · similarity 0.92)
- Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models (ICLR 2025 · similarity 0.90)
- MAGIC: Generating Self-Correction Guideline for In-Context Text-to-SQL (AAAI 2025 · similarity 0.90)
- Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing (NeurIPS 2024 · similarity 0.90)
- A Theoretical Understanding of Self-Correction through In-context Alignment (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.