Research map / Multi-agent systems and games / LLM agents
Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a World-Model-Augmented (WMA) web agent that trains an LLM (Llama-3.1-8B) to predict transition-focused natural-language descriptions of next web observations, and uses it with a value function to select actions among sampled candidates without training the policy.
Problem
LLM-based web agents perform poorly on long-horizon tasks because they lack world models to foresee the outcomes of actions, leading to irreversible mistakes; training LLMs to predict full next observations (long HTML/accessibility trees with many repeated elements) is inefficient.
Approach
Abstract raw observations into free-form descriptions of state transitions (using Hungarian-matching of element differences), train an LLM world model to predict these abstractions from (instruction, observation, action), and at inference simulate the outcome of each sampled action candidate and pick the one with the highest value-function score.
Key results
- On WebArena with GPT-4o-mini policy, WMA raises overall SR from 9.4% (vanilla CoT) to 13.5% (+43.6%); with GPT-4o, WMA reaches 16.6% vs 13.1% for CoT (+29.7%) versus tree search's 19.2% (+28.0%).
- On Mind2Web cross-task, AWM+WMA achieves 25.4% SR vs 15.3% for AWM (w/o EF).
- WMA is 5.3x faster (140.3 vs 748.3 s per instance) and 6.8x cheaper (API cost $0.4 vs $2.7) than tree search agent.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add simulated action outcomes and value-based selection | Vanilla CoT with GPT-4o · WebArena | success rate: 13.1% -> 16.6% | Helped |
| Add simulated action outcomes and value-based selection | Vanilla CoT with GPT-4o-mini · WebArena | success rate: 9.4% -> 13.5% | Helped |
| Add WMA to AWM | AWM · Mind2Web Cross-Task | SR: 15.3% -> 25.4% | Helped |
| Add WMA to AWM | AWM · Mind2Web Cross-Website | SR: 6.2% -> 8.5% | Helped |
| Add WMA to AWM | AWM · Mind2Web Cross-Domain | SR: 9.5% -> 10.1% | Helped |
| Add simulated next states to reward estimation (ablation) | WMA · WebArena overall (200-instance ablation) | success rate: 18.0% -> 22.0% | Helped |
| Train the world model (ablation) | WMA · WebArena overall (200-instance ablation) | success rate: 17.5% -> 22.0% | Helped |
| Use transition-focused observation abstraction instead of full observations (ablation) | WMA world model · WebArena overall (200-instance ablation) | success rate: 14.5% -> 22.0% | Helped |
| Fine-tune the value function instead of using prompted GPT-4o-mini (ablation) | WMA · WebArena | success rate: 12.7% -> 13.5% | Helped |
| Add image input to the Qwen2-VL-2B world model (ablation) | AWM+WMA · Mind2Web Cross-Task | SR: 23.7 -> 36.7 | Helped |
| Increase search depth from 1 to 3 at width 3 (ablation) | WMA multi-step search · WebArena | success rate: 17.1 -> 19.6 | Helped |
| Allocate an equal exploration budget to width rather than depth (ablation) | WMA multi-step search · WebArena | success rate: 19.6 vs. 20.5 | Mixed |
Surprising findings
- Current LLMs (e.g., Claude-3.5-Sonnet) perform near random at predicting the next state caused by their actions, with average accuracy of 54.75% across adopted LLMs.
- Element filtering from MindAct substantially hurts AWM performance on Mind2Web, so results are reported without it.
Limitations
- Errors in world model predictions: counterfactual imagination (42%), low competence with web elements (26%), overly generic statements (24%).
- Gains on WebArena absolute SR are modest (e.g., 16.6% with GPT-4o, below tree search's 19.2%); Shopping shows small improvement due to large state space.
- Multi-step search suffers from error accumulation and element ID changes between search and execution.
Experimental setup
Datasets: WebArena Mind2Web
Benchmarks: WebArena Mind2Web
Metrics: success rate step success rate element accuracy action F1 API cost inference time coverage
Baselines: Vanilla CoT AutoEval BrowserGym SteP AWM Tree search agent HTML-T5-XL MindAct Synapse
Builds on: Agent Workflow Memory Tree search agent Dreamer MindAct
Closest papers by meaning
- AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents (ICLR 2025 · similarity 0.90)
- Test-Time Adaptation for LLM Agents via Environment Interaction (ICLR 2026 · similarity 0.90)
- A Subgoal-driven RL Framework for Improving Long-Horizon Web Agents (NeurIPS 2026 · similarity 0.90)
- WebArena: A Realistic Web Environment for Building Autonomous Agents (ICLR 2024 · similarity 0.90)
- REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites (NeurIPS 2025 · similarity 0.89)
- WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.