Research map / Large language models / Agents, code and math
Position: LLMs Can’t Plan, But Can Help Planning in LLM-Modulo Frameworks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Position paper arguing that auto-regressive LLMs cannot plan or self-verify on their own, and proposing LLM-Modulo frameworks where LLMs generate candidates and act as knowledge sources while external sound model-based critics verify them.
Problem
Whether LLMs can perform planning and self-verification by themselves, and what constructive role they can play in planning/reasoning tasks given their unreliability.
Approach
Reframe LLMs as approximate knowledge sources and candidate generators inside a generate-test-critique loop where external sound model-based verifiers (critics, e.g. VAL for PDDL) vet every candidate, with a backprompt controller pooling critiques, rather than using LLMs as autonomous planners or simple format translators.
Key results
- Table results show state-of-the-art LLMs (e.g., Claude-3-Opus 48.17% one-shot, 59.3% zero-shot) solve only a fraction of Blocksworld instances and near 0% on Mystery Blocksworld.
- With VAL back-prompting over up to 15 rounds, LLM-Modulo raises Blocksworld accuracy to about 82% and Logistics to about 70% (as reported in prior work).
- In travel planning, LLM-Modulo with automated critics reportedly improves performance about 6x over baselines with 10 backprompt cycles, including with GPT-3.5-turbo.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LLM-Modulo with VAL back prompting | LLM plan generation · Blocksworld | accuracy: 82\% | Helped |
| Use LLM-Modulo with VAL back prompting | LLM plan generation · Logistics | accuracy: 70\% | Helped |
| Use LLM-Modulo with VAL back prompting | LLM plan generation · Mystery BW | accuracy: about 10\% | Mixed |
| Add automated critics and back prompting | GPT-3.5-Turbo travel planning · Travel planning benchmark | performance rate: 6x of baselines | Helped |
| Use Chain of Thought and ReAct prompt strategies | GPT-3.5-Turbo travel planning · Travel planning benchmark | performance rate: 0.7\% | Hurt |
| Use LLMs in autonomous mode to generate executable plans | GPT-4 · Planning problem instances | executable and goal-reaching plans: only about 12\% | Hurt |
| Have an LLM self-critique and refine its candidate solutions | GPT-4 graph coloring · Graph Coloring | performance | Hurt |
| Obfuscate action and object names | LLM plan generation · Planning domains | executable-plan performance | Hurt |
Surprising findings
- Self-critiquing by LLMs on graph coloring performs worse than no critique because the LLM fails to recognize correct colorings it generated.
- Obfuscating action and object names degrades LLM planning performance while standard planners are unaffected, suggesting approximate retrieval rather than planning.
Limitations
- LLM-Modulo guarantees soundness only through external hard critics; completeness depends on LLM candidate generation.
- Case study results are preliminary and largely reported from prior work rather than newly evaluated here.
- Requires human effort for domain model curation and verifier construction.
Experimental setup
Datasets: PlanBench Blocksworld Mystery Blocksworld Logistics travel planning benchmark
Benchmarks: PlanBench Blocksworld Mystery Blocksworld Logistics travel planning benchmark
Metrics: fraction of instances correct pass rate
Baselines: GPT-4 Tree of Thoughts Chain of Thought ReAct
Builds on: SAT Modulo Theories Tree of Thoughts FunSearch AlphaGeometry PlanBench
Closest papers by meaning
- Can LLMs Fix Issues with Reasoning Models? Towards More Likely Models for AI Planning (AAAI 2024 · similarity 0.90)
- Planning in the Dark: LLM-Symbolic Planning Pipeline Without Experts (AAAI 2025 · similarity 0.89)
- LLMs Can Plan Only If We Tell Them (ICLR 2025 · similarity 0.89)
- On the self-verification limitations of large language models on reasoning and planning tasks (ICLR 2025 · similarity 0.89)
- Classical Planning with LLM-Generated Heuristics: Challenging the State of the Art with Python Code (NeurIPS 2025 · similarity 0.88)
- Large Language Models as Planning Domain Generators (Student Abstract) (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.