Research map / Multi-agent systems and games / Human-AI and goal-conditioned
World-model-based agents
Builds learned or code-based world models for planning and generalist agent behavior.
110 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Human-AI and goal-conditioned, keyword cluster “environments · learn · exploration”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
22 papers in 2023–24 and 39 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 7.4% to 10.1%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Opponent Modeling in Deep Reinforcement Learning (ICML 2016 · 108 citations)
- 2017 · World of Bits: An Open-Domain Platform for Web-Based Agents (ICML 2017 · 36 citations)
- 2018 · VirtualHome: Simulating Household Activities via Programs (CVPR 2018 · 374 citations)
- 2019 · Learning Latent Dynamics for Planning from Pixels (ICML 2019 · 339 citations)
- 2020 · Emergent Tool Use From Multi-Agent Autocurricula (ICLR 2020 · 372 citations)
- 2021 · Mastering Atari with Discrete World Models (ICLR 2021 · 22 citations)
- 2022 · MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge (NeurIPS 2022 · 59 citations)
- 2023 · Describe, Explain, Plan and Select: Interactive Planning with LLMs Enables Open-World Multi-Task Agents (NeurIPS 2023 · 17 citations)
- 2024 · Equity-Transformer: Solving NP-Hard Min-Max Routing Problems as Sequential Generation with Equity Context (AAAI 2024 · 18 citations)
- 2025 · A Domain-Independent Agent Architecture for Adaptive Operation in Evolving Open Worlds (AAAI 2025 · 5 citations)
- 2026 · OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft (ICML 2026 · 0 citations)
Earliest papers here
- Opponent Modeling in Deep Reinforcement Learning (ICML 2016 · 108 citations)
- A Framework for Resolving Open-World Referential Expressions in Distributed Heterogeneous Knowledge Bases (AAAI 2016 · 31 citations)
- Efficient state-space modularization for planning: theory, behavioral and neural signatures (NeurIPS 2016 · 20 citations)
What has been tried
80 papers here report 767 measured changes (a component added, removed or swapped, compared with a baseline): helped 70%, no clear effect 7%, hurt 18%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add multi-agent positional encoding and context encoder | mTSP (N=100, M=10) · cost | 2.12 -> 1.96 | Helped | Equity-Transformer: Solving NP-Hard Min-Max Routing Problems as Sequential Generation with Equity Context (AAAI 2024 · 18 citations) |
| Add Planner and Verifier modules to the Actor | MAP-THOR, 2 agents · success rate | 0.33 -> 0.45 | Helped | Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments (NeurIPS 2024 · 7 citations) |
| Use the optimism objective φ2 instead of data consistency alone | MiniGrid-UnlockPickup-v0 · actions to learn the correct world model | 3×10^8 vs no more than 100 | Helped | WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment (NeurIPS 2024 · 5 citations) |
| Remove task knowledge | ALFWorld and ScienceWorld, seen and unseen · average reward | Hurt | Agent Planning with World Knowledge Model (NeurIPS 2024 · 4 citations) | |
| add transfer learning between tasks instead of training each task from scratch | 25 tasks, 5 replications · success rate | 36.8\% -> 63.2\% | Helped | OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code (ICLR 2025 · 3 citations) |
| add object sampling | 18-domain benchmark suite · normalized coverage | 17.74 -> 16.99 (drop -0.743 when disabled) | Helped | PRP Rebooted: Advancing the State of the Art in FOND Planning (AAAI 2024 · 3 citations) |
| Represent trajectories as natural language instead of programs | MiniWoB++ · task success rate | Hurt | Synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scale (NeurIPS 2024 · 3 citations) | |
| Add token input skip connection | Platformers-200/50 test set · PSNR | 26.36 -> 26.65 | Helped | Exploration-Driven Generative Interactive Environments (CVPR 2025 · 2 citations) |
| Add object rotation augmentation | RLBench close jar · success rate | 10.0 ± 3.10 -> 25.2 ± 0.98 | Helped | Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imagination (ICLR 2025 · 2 citations) |
| Use 3 denoising steps instead of 1 | Atari 100k, 10 highest-performing games · Mean HNS | 1.962 -> 3.052 | Helped | Diffusion for World Modeling: Visual Details Matter in Atari (NeurIPS 2024 · 2 citations) |
| Add the dynamics residual term | dynamics ablation maps · prediction error | 0.434 ± 0.351 -> 0.016 ± 0.023 | Helped | Grounded Answers for Multi-agent Decision-making Problem through Generative World Model (NeurIPS 2024 · 2 citations) |
| add TSD regularization to PBT | Predator Prey, Unseen-Avg. · Avg. Ret. | Helped | An Open-Ended Learning Framework for Opponent Modeling (AAAI 2025 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 26 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Baselines
- PPO · 3 of 26 papers
- LEAP · 2 of 26 papers
- Decision Transformer · 2 of 26 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 61 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 4 papers
- Carnegie Mellon University · 4 papers
- Peking University · 4 papers
- Tencent · 3 papers
- University of Chinese Academy of Sciences · 3 papers
- Google DeepMind · 3 papers
- Amazon · 3 papers
- Technion - Israel Institute of Technology · 2 papers
- Korea Advanced Institute of Science and Technology · 2 papers
- Chinese Academy of Sciences · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Language-grounded reinforcement agents (96 papers)
- Imitation and skill learning (100 papers)
- Intrinsically motivated exploration (86 papers)
Elsewhere in the topic
- Human-AI coordination agents (125 papers)
- Embodied agent learning (93 papers)
- Boundedly rational agent modeling (87 papers)
- Collaborative multi-agent perception (86 papers)
- Interactive multi-agent trajectory prediction (75 papers)
- Learning-guided MAPF search (73 papers)
- Theory-of-mind social reasoning (69 papers)
- Vision-language navigation agents (66 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).