Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Intrinsically motivated exploration
Drives agents with curiosity and information-based intrinsic rewards to learn skills without external reward.
86 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Human-AI and goal-conditioned, keyword cluster “environments · learn · exploration”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
19 papers in 2023–24 and 21 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 6.4% to 5.4%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Curiosity-driven Exploration by Self-supervised Prediction (ICML 2017 · 1,773 citations)
- 2018 · Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces (AAAI 2018 · 153 citations)
- 2019 · CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learning (ICML 2019 · 41 citations)
- 2020 · Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration (NeurIPS 2020 · 54 citations)
- 2021 · Relative Variational Intrinsic Control (AAAI 2021 · 12 citations)
- 2022 · Curiosity-Driven Exploration via Latent Bayesian Surprise (AAAI 2022 · 30 citations)
- 2023 · Human-level Atari 200x faster (ICLR 2023 · 6 citations)
- 2024 · Robust agents learn causal world models (ICLR 2024 · 5 citations)
- 2025 · Position: Truly Self-Improving Agents Require Intrinsic Metacognitive Learning (ICML 2025 · 7 citations)
- 2026 · Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration? (ICLR 2026 · 1 citation)
Earliest papers here
- Curiosity-driven Exploration by Self-supervised Prediction (ICML 2017 · 1,773 citations)
- Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning (ICLR 2017 · 106 citations)
- Learning to Perform Physics Experiments via Deep Reinforcement Learning (ICLR 2017 · 56 citations)
What has been tried
67 papers here report 674 measured changes (a component added, removed or swapped, compared with a baseline): helped 57%, no clear effect 11%, hurt 24%, mixed 8%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| combine at least two of agent values, critic, and trust rather than a single mechanism | 4 MuJoCo environments (averaged) · normalized reward over time | Mixed | Peer Learning: Learning Complex Policies in Groups from Scratch via Action Recommendations (AAAI 2024 · 4 citations) | |
| regularize language actions toward the world model's predicted next token | LangRoom, vocabulary size 10,000 · question-answer accuracy | Helped | Learning to Model the World With Language (ICML 2024 · 3 citations) | |
| Use the 3-successful-attempt cutoff instead of the 1-successful-attempt cutoff for learning a goal | Goal selection model comparison · winning model XP | 1 -> 1 | No clear effect | Latent Learning Progress Drives Autonomous Goal Selection in Human Reinforcement Learning (NeurIPS 2024 · 3 citations) |
| Add Q-learning fine-tuning on top of the skill machine (QL-SM vs zero-shot SM) | Office Gridworld · average return | Helped | Skill Machines: Temporal Logic Skill Composition in Reinforcement Learning (ICLR 2024 · 2 citations) | |
| Train with oracle accuracy that is neither too high nor too low | Memory Sequence · return | Mixed | Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning (NeurIPS 2024 · 2 citations) | |
| Add the inverse Laplace transform and time-cell representation | 3D interval timing · performance | Helped | Deep Reinforcement Learning with Time-Scale Invariant Memory (AAAI 2025 · 1 citation) | |
| Add adaptive skill selection across training stages | GRF 11 vs 11 · policy style balance | Helped | Vision-Based Generic Potential Function for Policy Alignment in Multi-Agent Reinforcement Learning (AAAI 2025 · 0 citations) | |
| Add the last 100 student history entries to the teacher | Minigrid General · success rate | 0.487 -> 0.527 | Helped | Heterogeneous Adversarial Play in Interactive Environments (NeurIPS 2025 · 0 citations) |
| add mask loss (intervention verification) | Chemical Fork and Chain OOD states (n=2, 4, 6) · prediction accuracy | Fork n=6: 46.04±8.56 -> 50.04±8.56; Chain n=2: 47.75±7.52 -> 51.99±6.58; Fork n=4: 51.70±8.58 -> 50. | Mixed | Curious Causality-Seeking Agents Learn Meta Causal World (NeurIPS 2025 · 0 citations) |
| Replace uniform random action selection with intelligent action selection | Game of 24 · success rate (%) | \phantom{0}68 ± \phantom{0}9.0 -> \textbf{100 ± \phantom{0}0.0} | Helped | Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models (ICLR 2025 · 0 citations) |
| add Gaussian action noise to BC | Libero task hidden · success rate | 0.590 { ± 0.013} -> 0.684 { ± 0.013} | Helped | Behavioral Exploration: Learning to Explore via In-Context Adaptation (ICML 2025) |
| Use future transitions as privileged information instead of hidden-state privileged information | Tiger Door · return | No clear effect | Learning to Explore in POMDPs with Informational Rewards (ICML 2024) |
What reviewers will expect
Datasets and baselines named in the 14 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Baselines
- ICM · 2 of 14 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 40 papers; from OpenAlex and the papers' affiliations).
- Google DeepMind · 7 papers
- Google · 5 papers
- University of California, Berkeley · 4 papers
- University College London · 3 papers
- Chinese Academy of Sciences · 2 papers
- University of Chinese Academy of Sciences · 2 papers
- Peking University · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Language-grounded reinforcement agents (96 papers)
- World-model-based agents (110 papers)
- Imitation and skill learning (100 papers)
Elsewhere in the topic
- Human-AI coordination agents (125 papers)
- Embodied agent learning (93 papers)
- Boundedly rational agent modeling (87 papers)
- Collaborative multi-agent perception (86 papers)
- Interactive multi-agent trajectory prediction (75 papers)
- Learning-guided MAPF search (73 papers)
- Theory-of-mind social reasoning (69 papers)
- Vision-language navigation agents (66 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).