Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Agent Planning with World Knowledge Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a parametric World Knowledge Model (WKM), a LoRA-trained knowledge model that self-synthesizes global task knowledge and local state knowledge from expert and sampled trajectories to guide LLM agent planning.
Problem
LLM agents performing interactive planning suffer from blind trial-and-error in global planning and hallucinatory actions in local planning due to weak understanding of the environment.
Approach
Imitating human mental world knowledge, a separately LoRA-trained knowledge model generates instance-level task knowledge (contrasting expert vs. sampled trajectories) as a prior and state knowledge that is used as a retrieval key into a state knowledge base; the retrieved next-action distribution is combined with the agent's action probabilities.
Key results
- WKM on Mistral-7B reaches 73.57 seen / 76.87 unseen on ALFWorld, versus 44.29 / 38.05 for GPT-4 with ReAct, and beats the best fine-tuned baseline ETO by +3.13 and +5.44.
- WKM on Llama-3-8B improves WebShop to 66.64 versus 62.76 for GPT-4.
- Hallucinatory action rate on ALFWorld unseen tasks drops from 36.57% (ETO) to 29.85% for WKM with Mistral-7B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add task and retrieved state knowledge with WKM | Mistral-7B agent · ALFWorld Seen | average reward: 70.44 -> 73.57 | Helped |
| Add task and retrieved state knowledge with WKM | Mistral-7B agent · ALFWorld Unseen | average reward: 71.43 -> 76.87 | Helped |
| Add task and retrieved state knowledge with WKM | Gemma-7B agent · WebShop | average reward: 62.67 -> 63.75 | Helped |
| Add task and retrieved state knowledge with WKM | Llama-3-8B agent · ScienceWorld Unseen | average reward: 52.33 -> 54.75 | Helped |
| Remove task knowledge (ablation) | WKM with Mistral-7B · ALFWorld and ScienceWorld, seen and unseen | average reward | Hurt |
| Remove state knowledge (ablation) | WKM with Mistral-7B · ALFWorld and ScienceWorld, seen and unseen | average reward | Hurt |
| Synthesize task knowledge from chosen trajectories only, excluding rejected trajectories (ablation) | WKM · Planning tasks | average reward | Hurt |
| Set γ=0 and fully trust the state knowledge base (ablation) | WKM with Mistral-7B · ALFWorld, WebShop, and ScienceWorld | average reward: 73.57 -> 1.58; 76.87 -> 0.00; 65.48 -> 25.83; 62.12 -> 18.69; 53.62 -> 15.37 | Hurt |
| Remove state knowledge by setting γ=1 (ablation) | WKM with Mistral-7B · ALFWorld, WebShop, and ScienceWorld | average reward: 73.57 -> 69.29; 76.87 -> 75.37; 65.48 -> 63.68; 62.12 -> 60.81; 53.62 -> 53.42 | Mixed |
| Use explicit state knowledge in the agent context instead of retrieving from a state knowledge base (ablation) | WKM agent · Planning tasks | average reward | Hurt |
| Use dataset-level human-designed task knowledge instead of instance-level generated task knowledge (ablation) | WKM without state knowledge · ALFWorld and ScienceWorld, especially unseen tasks | average reward | Hurt |
| Use Mistral-7B WKM task knowledge to guide GPT-4 | GPT-4 on ALFWorld · ALFWorld Seen and Unseen | average reward: 44.29 -> 50.71; 38.05 -> 47.01 | Helped |
Surprising findings
- Explicit state knowledge in the context performs worse than omitting state knowledge entirely, so implicit retrieval-based constraints work better.
- A weak Mistral-7B knowledge model providing task knowledge improves stronger agents GPT-3.5-Turbo and GPT-4 (e.g., GPT-4 ALFWorld seen 44.29 to 50.71).
Limitations
- Knowledge is limited to textual information; no multimodal world knowledge.
- The world knowledge model cannot update dynamically with environment changes or agent feedback.
- Generating knowledge and retrieval adds inference overhead.
Experimental setup
Datasets: ALFWorld WebShop ScienceWorld
Benchmarks: ALFWorld WebShop ScienceWorld
Metrics: average reward average steps hallucinatory action rate
Baselines: ReAct Reflexion NAT ETO KnowAgent GPT-3.5-Turbo GPT-4
Builds on: ETO KnowAgent ReAct Reflexion NAT
Closest papers by meaning
- World-aware Planning Narratives Enhance Large Vision-Language Model Planner (NeurIPS 2025 · similarity 0.89)
- From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents (ICLR 2026 · similarity 0.89)
- WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment (NeurIPS 2024 · similarity 0.89)
- Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments (NeurIPS 2024 · similarity 0.89)
- R2C: Mapping Room to Chessboard to Unlock LLM As Low-Level Action Planner (CVPR 2025 · similarity 0.88)
- Agent2World: Learning to Generate Symbolic World Models via Adaptive Multi-Agent Feedback (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.