Research map / Large language models / Agents, code and math
Leveraging Environment Interaction for Automated PDDL Translation and Planning with Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an LLM-based iterative tree search that generates PDDL problem and domain files using environment feedback and a new Exploration Walk metric, with no human intervention.
Problem
Generating accurate PDDL domain and problem files from natural language normally needs human correction, and small domain errors make classical planning fail with no useful feedback.
Approach
Exploration Walk (EW) is a smooth similarity metric between an LLM-generated domain and the true environment, computed by checking executability of random legal action sequences in both directions (harmonic mean). EW scores guide tree-search refinement of the domain and provide natural-language feedback without ground-truth PDDL.
Key results
- Average task solve rate of 0.66 for P&D Tree (n_d=10, n_p=5) vs 0.29 for intrinsic CoT planning with GPT-4 and 0.34 for P&D Chain.
- The method solves 7 of 10 domains (domain-level success > 0.5), compared to 3 for intrinsic CoT.
- Average EW score of 0.84 for P&D Tree, vs 0.53 for P&D Chain.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use iterative EW-guided P&D Tree refinement with multiple problem and domain candidates instead of intrinsic chain-of-thought planning | GPT-4 planning · 10-domain average | tasks solved: 0.29 -> 0.66 | Helped |
| Use iterative EW-guided P&D Tree refinement with multiple candidates instead of single-candidate P&D Chain (ablation) | GPT-4 PDDL translation and planning · 10-domain average | tasks solved: 0.34 -> 0.66 | Helped |
| Use iterative EW-guided P&D Tree refinement with multiple candidates instead of single-candidate P&D Chain (ablation) | GPT-4 PDDL translation and planning · 10-domain average | Exploration Walk score: 0.53 -> 0.84 | Helped |
| Add domain proposals for each problem before generating its PDDL (ablation) | P&D Tree · 10-domain average | tasks solved: 0.66 -> 0.60 | Hurt |
| Add domain proposals for each problem before generating its PDDL (ablation) | P&D Tree · 10-domain average | Exploration Walk score: 0.84 -> 0.85 | Helped |
| Use P&D Tree instead of intrinsic chain-of-thought planning | GPT-4 planning · Floortile | tasks solved: 0.00 -> 0.90 | Helped |
| Use P&D Tree instead of intrinsic chain-of-thought planning | GPT-4 planning · Grippers | tasks solved: 0.60 -> 1.00 | Helped |
| Use P&D Tree instead of intrinsic chain-of-thought planning | GPT-4 planning · Grippers-ood | tasks solved: 0.30 -> 0.70 | Helped |
| Use P&D Tree instead of intrinsic chain-of-thought planning | GPT-4 planning · Hiking | tasks solved: 0.00 -> 1.00 | Helped |
| Use P&D Tree instead of intrinsic chain-of-thought planning | GPT-4 planning · Miconic | tasks solved: 1.00 -> 1.00 | No clear effect |
| Use P&D Tree instead of intrinsic chain-of-thought planning | GPT-4 planning · Movie | tasks solved: 1.00 -> 1.00 | No clear effect |
| Use P&D Tree instead of intrinsic chain-of-thought planning | GPT-4 planning · Barman | tasks solved: 0.00 -> 0.00 | No clear effect |
Surprising findings
- Small predicate-design errors (e.g., omitting an argument in free or used) can make the final domain entirely wrong and cause plan search to fail.
- Intrinsic CoT planning solves only 0.29 of tasks on average, and fails completely on several domains such as Barman and Childsnack.
Limitations
- Random exploration walks may explore insufficiently and inefficiently, raising cost.
- Only applied to PDDL environments, not digital or physical ones.
- Perfect task solve rate does not guarantee exact equivalence to the ground-truth domain.
Experimental setup
Datasets: PDDL domains from International Planning Competition and prior PDDL benchmarks (nine domains plus Grippers-ood) GPT-4 back-translated natural language descriptions
Benchmarks: Barman Childsnack Driverlog Floortile Grippers Grippers-ood Hiking Miconic Movie Termes
Metrics: task solve rate (Best@4) exploration walk score
Baselines: Intrinsic planning (CoT) Intrinsic planning (No CoT) P&D Chain
Builds on: Chain-of-thought prompting Tree of Thoughts LLM+P Fast Downward
Closest papers by meaning
- Generalized Planning in PDDL Domains with Pretrained Large Language Models (AAAI 2024 · similarity 0.90)
- Large Language Models as Planning Domain Generators (Student Abstract) (AAAI 2024 · similarity 0.89)
- Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks (ICML 2025 · similarity 0.89)
- LLMs Can Plan Only If We Tell Them (ICLR 2025 · similarity 0.89)
- SayCanPay: Heuristic Planning with Large Language Models Using Learnable Domain Knowledge (AAAI 2024 · similarity 0.89)
- LexiCon: a Benchmark for Planning under Temporal Constraints in Natural Language (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.