Research map / Large language models / Agents, code and math
LLM+AL: Bridging Large Language Models and Action Languages for Complex Reasoning About Actions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LLM+AL, which pairs an LLM (o1-preview) that generates BC+ action-language signatures, knowledge, rules and queries with the cplus2asp reasoner and LLM self-revision, evaluated on Missionaries and Cannibals elaborations and puzzle variations.
Problem
LLMs struggle with complex action reasoning requiring systematic search, and PDDL is limited in expressing knowledge about actions such as state constraints, indirect effects, and defaults.
Approach
Use the LLM to translate natural-language problems into BC+ action-language programs (signature, commonsense and domain knowledge, causal laws, queries), then run the BC+ reasoner and let the LLM self-revise using solver feedback and LLM-generated sample queries, with minimal human correction.
Key results
- LLM+AL automatically solves 7 MCP problems and, with an average of 3.1 manual corrections, the rest; baselines solve at most 7 (o1-preview) and ChatGPT-4 solves 3.
- On puzzle variations, LLM+AL solves 8 of 12 automatically plus 5 with human intervention, versus 8+1 for ChatGPT-4+Code and 5 for o1-preview.
- Self-revision raises executable programs from 22.3% (7/30) to 86.7% (26/30) and correct-answer programs from 16.6% to 50%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LLM+AL instead of standalone LLMs | complex action reasoning · MCP elaborations | problems solved correctly: 7 automatically; ChatGPT-4: 3; Claude 3 Opus: 0; Gemini 1.0 Ultra: 0 | Helped |
| Use LLM+AL instead of standalone LLMs | complex action reasoning · puzzles and variations | problems solved correctly: 8+5△; ChatGPT-4: 5; o1-preview: 5 | Helped |
| Add Self-Revision with BC+ reasoner feedback (ablation) | LLM+AL · 30 problems | programs executable without syntax errors: 22.3\% (7/30) -> 86.7\% (26/30) | Helped |
| Add Self-Revision with BC+ reasoner feedback (ablation) | LLM+AL · 30 problems | programs producing correct answers: 16.6\% (5/30) -> 50\% (15/30) | Helped |
| Add Self-Revision with BC+ reasoner feedback (ablation) | LLM+AL · 30 problems | program issues: 70 -> 42 | Helped |
| Provide human corrections to the answer | ChatGPT-4 · MCP elaborations | correctness: 50 iterations | No clear effect |
Surprising findings
- Standalone LLMs did not improve even after 50 rounds of human feedback with ChatGPT-4.
- o1-preview frequently claimed no solution exists for solvable instances (five false negatives).
Limitations
- Occasionally requires human corrections to generated BC+ programs.
- Does not guarantee non-existence of plans of arbitrary length.
- Evaluated on a small set of puzzles, with baselines run in May 2024 and o1-preview in September 2024.
Experimental setup
Datasets: Missionaries and Cannibals elaborations (McCarthy) classic puzzles with authors' variations (River Cross, Tower of Hanoi, Sudoku)
Benchmarks: Missionaries and Cannibals elaborations River Cross Tower of Hanoi variations Sudoku variations
Metrics: number of problems solved correctly number of manual corrections issue counts before and after self-revision
Baselines: ChatGPT-4 Claude 3 Opus Gemini 1.0 Ultra o1-preview ChatGPT-4+Code
Builds on: BC+ cplus2asp Program-of-Thought PDDL-based LLM planning
Closest papers by meaning
- Distilling LLM Agent into Small Models with Retrieval and Code Tools (NeurIPS 2025 · similarity 0.90)
- DynaAct: Large Language Model Reasoning with Dynamic Action Spaces (NeurIPS 2025 · similarity 0.89)
- Enhancing Decision-Making of Large Language Models via Actor-Critic (ICML 2025 · similarity 0.89)
- Planning in the Dark: LLM-Symbolic Planning Pipeline Without Experts (AAAI 2025 · similarity 0.89)
- From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information? (ICML 2025 · similarity 0.89)
- Large Language Models Are Neurosymbolic Reasoners (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.