Research map / Reinforcement learning / Visual and representation RL
Novelty and impact-driven exploration
Reward visiting novel or impactful states, or diversify behavior, to explore better.
85 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Visual and representation RL, keyword cluster “exploration · intrinsic · skills”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
23 papers in 2023–24 and 20 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 4.0% to 2.9%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Unifying Count-Based Exploration and Intrinsic Motivation (NeurIPS 2016)
- 2017 · EX2: Exploration with Exemplar Models for Deep Reinforcement Learning (NeurIPS 2017 · 59 citations)
- 2018 · GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms (ICML 2018 · 76 citations)
- 2019 · Exploration by random network distillation (ICLR 2019 · 245 citations)
- 2020 · Never Give Up: Learning Directed Exploration Strategies (ICLR 2020 · 72 citations)
- 2021 · Fast active learning for pure exploration in reinforcement learning (ICML 2021 · 25 citations)
- 2022 · Improving Intrinsic Exploration with Language Abstractions (NeurIPS 2022 · 16 citations)
- 2023 · Latent exploration for Reinforcement Learning (NeurIPS 2023 · 7 citations)
- 2024 · Explore 3D Dance Generation via Reward Model from Automatically-Ranked Demonstrations (AAAI 2024 · 4 citations)
- 2025 · Hyper: Hyperparameter Robust Efficient Exploration in Reinforcement Learning (ICML 2025 · 1 citation)
- 2026 · Task-Aware Exploration via a Predictive Bisimulation Metric (ICML 2026 · 0 citations)
Earliest papers here
- Unifying Count-Based Exploration and Intrinsic Motivation (NeurIPS 2016)
- VIME: Variational Information Maximizing Exploration (NeurIPS 2016)
- EX2: Exploration with Exemplar Models for Deep Reinforcement Learning (NeurIPS 2017 · 59 citations)
What has been tried
69 papers here report 721 measured changes (a component added, removed or swapped, compared with a baseline): helped 61%, no clear effect 13%, hurt 20%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Inject progressively higher ε-greedy noise into the BC policy | AIST++ training music · FID_k | 13.94 -> 40.45 -> 48.59 -> 53.79 -> 57.18 | Hurt | Explore 3D Dance Generation via Reward Model from Automatically-Ranked Demonstrations (AAAI 2024 · 4 citations) |
| replace the SI2E embedding principle with the Dynamic Bottleneck | MetaWorld and DMControl tasks · final performance | Hurt | Effective Exploration Based on the Structural Information Principles (NeurIPS 2024 · 3 citations) | |
| remove forward dynamics learning | ObstructedMaze-2Q · return | 0.78 ± 0.182 -> 0.77 ± 0.245 | No clear effect | Abstract and Explore: A Novel Behavioral Metric with Cyclic Dynamics in Reinforcement Learning (AAAI 2024 · 2 citations) |
| replace information gain with RND intrinsic reward | State-based control tasks · performance | Hurt | MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization (ICLR 2025 · 1 citation) | |
| Use switching threshold 0.50 instead of 1.00 | 2D Navigation · mean episodic return | 0.407 ± 0.055 -> 0.358 ± 0.151 | Hurt | KEA: Keeping Exploration Alive by Proactively Coordinating Exploration Strategies (ICML 2025 · 0 citations) |
| Combine semantic reward with disagreement/information-gain reward rather than using semantic reward alone | KeyRoom-S15 and KeyChest · task-relevant interactions and task rewards | Helped | SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Models (ICML 2025 · 0 citations) | |
| Replace latent dynamics with vanilla dynamics | Humanoid-run · TAR | 489 ± 9 -> 408 ± 35 | Hurt | Off-policy Reinforcement Learning with Model-based Exploration Augmentation (NeurIPS 2025 · 0 citations) |
| Use ETD instead of Euclidean distance in inverse-dynamics or discriminator representations | MiniGrid ObstructedMaze-1Q · exploration efficiency | Helped | Episodic Novelty Through Temporal Distance (ICLR 2025 · 0 citations) | |
| Add multi-goal actor training rather than training the actor only on the single hard goal | Single-goal data collection · performance | Helped | A Single Goal is All You Need: Skills and Exploration Emerge from Contrastive RL without Rewards, Demonstrations, or Subgoals (ICLR 2025 · 0 citations) | |
| Replace AP with CASM | DM-HARD-8 · capped human-normalized score | Helped | Unlocking the Power of Representations in Long-term Novelty-based Exploration (ICLR 2024 · 0 citations) | |
| change the latent-vector sampling distribution to Uniform([-0.5, 0.5]^d) or isotropic normal | Atari and IsaacGym · performance | No clear effect | Random Latent Exploration for Deep Reinforcement Learning (ICML 2024 · 0 citations) | |
| add a separated world model | Quadruped, 500k pre-training frames · normalized return | 0.15 -> 0.49 | Helped | Leveraging Separated World Model for Exploration in Visually Distracted Environments (NeurIPS 2024 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 12 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MiniGrid · 3 of 12 papers
- DeepMind Control Suite · 2 of 12 papers
- OpenAI Gym · 2 of 12 papers
Baselines
- SAC · 3 of 12 papers
- PPO · 3 of 12 papers
- Plan2Explore · 2 of 12 papers
- TD3 · 2 of 12 papers
- RND · 2 of 12 papers
- E3B · 2 of 12 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 43 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 3 papers
- Max Planck Institute for Intelligent Systems · 3 papers
- Google DeepMind · 3 papers
- Université de Montréal · 3 papers
- University of Montreal · 3 papers
- Meta · 3 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Hierarchical skills and priors (84 papers)
- Reward shaping for sparse rewards (51 papers)
- Unsupervised skill discovery (45 papers)
Elsewhere in the topic
- In-context and adaptive RL (202 papers)
- Scalable RL systems and environments (179 papers)
- Probabilistic and autonomous RL (177 papers)
- RL for combinatorial routing (158 papers)
- Learned state and action representations (148 papers)
- Dexterous manipulation transfer (141 papers)
- RL with tree search for discovery (121 papers)
- Inferring decision-making from behavior (93 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).