Research map / Multi-agent systems and games / Human-AI and goal-conditioned
General agents need world models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves that any goal-conditioned agent satisfying a regret bound on multi-step composite (LTL-style) goals must implicitly encode a predictive world model of its environment, and gives an algorithm to recover the transition function from the policy.
Problem
Whether world models are necessary for flexible, goal-directed behaviour, or whether model-free learning suffices for general agents.
Approach
Formal reduction: a goal-conditioned policy with bounded regret on sequences of either-or composite goals reveals, via goal-switching behaviour, the environment transition probabilities with error bounded by regret and goal depth. Agents that are myopic (depth-1 goals only) provably cannot reveal transition probabilities.
Key results
- Theorem 1: world-model error scales as O(delta/sqrt(n)) + O(1/n) for regret delta and goal depth n, so any delta<1 yields arbitrarily accurate world models for large n.
- Theorem 2: an optimal myopic agent (depth-1 goals) imparts only trivial bounds (epsilon=1) on transition probabilities.
- In a 20-state, 5-action synthetic cMP, recovered world-model error decays roughly as O(n^-1/2) with goal depth, even when worst-case regret is 1 for some goals.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the agent's maximum goal depth | goal-conditioned agent and Algorithm 2 · Random cMP (20 states, 5 actions) | mean world-model error: ∼O(n−1/2) | Helped |
| Increase the agent's performance (decrease failure rate δ) | goal-conditioned agent · Controlled Markov processes | world-model error bound: O(δ/√n) + O(1/n) | Helped |
Surprising findings
- Myopic agents that only optimise immediate outcomes provably need not learn any world model, whereas multi-step goal agents must.
- Recovery works even when the agent violates the worst-case regret bound, as long as average regret is low.
Limitations
- Proof only covers fully observed Markov environments; partially observed or non-Markovian settings not addressed.
- Result shows existence of an encoded world model, not how the agent uses it or what it knows epistemically.
- Algorithm 1 may underestimate what an agent knows if it fails at planning (violates regret bound).
Experimental setup
Datasets: randomly generated controlled Markov process (20 states, 5 actions, sparse transitions)
Benchmarks: randomly generated cMP environment
Metrics: mean world-model error <epsilon> mean regret <delta> maximum goal depth N(<delta>=k)
Baselines: Algorithm 1 (world model recovery) Savage representation theorem mechanistic interpretability probing
Builds on: Richens & Everitt 2024 causal world models result Savage representation theorem goal-conditioned reinforcement learning linear temporal logic task specification
Closest papers by meaning
- WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment (NeurIPS 2024 · similarity 0.89)
- Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation (ICLR 2025 · similarity 0.89)
- Learning World Models for Unconstrained Goal Navigation (NeurIPS 2024 · similarity 0.88)
- Beyond Prediction: Steering VLM Agents with Retrospective World Modeling (NeurIPS 2026 · similarity 0.88)
- Agent Planning with World Knowledge Model (NeurIPS 2024 · similarity 0.88)
- Simplifying Latent Dynamics with Softly State-Invariant World Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.