Research map / Large language models / Reasoning and chain of thought
On the self-verification limitations of large language models on reasoning and planning tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Systematically tests GPT-4 self-verification and self-critique loops on Game of 24, Graph Coloring, and STRIPS planning, finding self-critique degrades performance while sound external verifiers improve it, and plain resampling with a sound verifier retains most gains.
Problem
Whether LLMs can self-critique and iteratively improve their own solutions on formally verifiable reasoning and planning tasks, given that verification is assumed easier than generation.
Approach
Decomposes self-critique into verification, critique generation, and critique consideration, and ablates each with a provably sound external verifier (SymPy, a graph checker, VAL) to show that gains come from sound verification and repeated guessing rather than from LLM critique content.
Key results
- LLM+LLM self-critique drops accuracy below standard prompting in most domains (e.g., Graph Coloring 16% standard vs 2% LLM+LLM; Game of 24 5% vs 3%).
- Sound-verifier feedback raises accuracy substantially (e.g., Blocksworld 40% to 87% with first-error feedback; Graph Coloring 37-38% with sound feedback).
- Pure resampling with a sound verifier (no critique, k=15) reaches 28% on Game of 24 and 40% on Graph Coloring, and k=25 reaches 42% and 44%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use iterative self-verification instead of standard prompting | GPT-4 · Game of 24 | accuracy: 5% -> 3% | Hurt |
| use iterative self-verification instead of standard prompting | GPT-4 · Graph Coloring | accuracy: 16% -> 2% | Hurt |
| use iterative self-verification instead of standard prompting | GPT-4 · Blocksworld | accuracy: 40% ->55% | Helped |
| use iterative self-verification instead of standard prompting | GPT-4 · Mystery Blocksworld | accuracy: 4% -> 0% | Hurt |
| replace self-verification with a sound verifier and first-error feedback (ablation) | GPT-4 · Game of 24 | accuracy: 5% -> 38% | Helped |
| replace self-verification with a sound verifier and first-error feedback (ablation) | GPT-4 · Graph Coloring | accuracy: 16% -> 37% | Helped |
| replace self-verification with a sound verifier and first-error feedback (ablation) | GPT-4 · Blocksworld | accuracy: 40% -> 87% | Helped |
| replace self-verification with a sound verifier and first-error feedback (ablation) | GPT-4 · Mystery Blocksworld | accuracy: 4% -> 8% | Helped |
| remove critique and repeat the same prompt with sound verification, k=25 (ablation) | GPT-4 · Game of 24 | accuracy: 5% -> 42% | Helped |
| remove critique and repeat the same prompt with sound verification, k=25 (ablation) | GPT-4 · Graph Coloring | accuracy: 16% -> 44% | Helped |
| remove critique and repeat the same prompt with sound verification, k=25 (ablation) | GPT-4 · Blocksworld | accuracy: 40% -> 72% | Helped |
| remove critique and repeat the same prompt with sound verification, k=25 (ablation) | GPT-4 · Mystery Blocksworld | accuracy: 4% -> 14% | Helped |
Surprising findings
- Resampling the same prompt with a sound verifier retains most of the gains of critique-based loops, suggesting critique content contributes little.
- Richer feedback (all errors vs first error) often did not improve and sometimes reduced performance in two domains.
Limitations
- Evaluated only on GPT-4 and 100 instances per domain.
- Game of 24 and graph coloring instances are small; prompt sensitivity explored only partially.
- Sound verifiers are domain-specific and not available in many real tasks.
Experimental setup
Datasets: 4nums.com Game of 24 instances Erdős–Rényi planar graphs (100, generated with GrinPy) Blocksworld and Mystery Blocksworld PDDL instances
Benchmarks: Game of 24 Graph Coloring Blocksworld Mystery Blocksworld
Metrics: accuracy false positive rate false negative rate critique accuracy
Baselines: Standard Prompting LLM+LLM self-critique Self-Consistency Tree of Thoughts CRITIC
Builds on: Tree of Thoughts Reflexion CRITIC self-consistency LLM-Modulo
Closest papers by meaning
- LLMs Can Plan Only If We Tell Them (ICLR 2025 · similarity 0.90)
- Graph-enhanced Large Language Models in Asynchronous Plan Reasoning (ICML 2024 · similarity 0.90)
- Chain of Thoughtlessness? An Analysis of CoT in Planning (NeurIPS 2024 · similarity 0.90)
- Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing (NeurIPS 2024 · similarity 0.90)
- Learning to Self-Verify Makes Language Models Better Reasoners (ICML 2026 · similarity 0.89)
- Self-Verifying Reflection Helps Transformers with CoT Reasoning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.