Research map / Reinforcement learning / Visual and representation RL
Unsupervised skill discovery
Discover diverse skills without reward using mutual-information and contrastive objectives.
45 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2019–2026) · an approach in Visual and representation RL, keyword cluster “exploration · intrinsic · skills”
State History Tried Baselines Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
17 papers in 2023–24 and 12 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 3.0% to 1.7%.
Open questions the data points to
- Papers report CIC on Walker Flip (score) anywhere from 631 to 715: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2019 · Diversity is All You Need: Learning Skills without a Reward Function (ICLR 2019 · 94 citations)
- 2020 · Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills (ICML 2020 · 21 citations)
- 2021 · APS: Active Pretraining with Successor Features (ICML 2021 · 12 citations)
- 2022 · Unsupervised Reinforcement Learning in Multiple Environments (AAAI 2022 · 9 citations)
- 2023 · Controllability-Aware Unsupervised Skill Discovery (ICML 2023 · 8 citations)
- 2024 · Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning (NeurIPS 2024 · 4 citations)
- 2025 · Skill Disentanglement in Reproducing Kernel Hilbert Space (AAAI 2025 · 0 citations)
- 2026 · Unsupervised Hierarchical Skill Discovery (ICML 2026 · 0 citations)
Earliest papers here
- Diversity is All You Need: Learning Skills without a Reward Function (ICLR 2019 · 94 citations)
- Composing Complex Skills by Learning Transition Policies (ICLR 2019 · 23 citations)
- Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills (ICML 2020 · 21 citations)
What has been tried
36 papers here report 330 measured changes (a component added, removed or swapped, compared with a baseline): helped 62%, no clear effect 13%, hurt 21%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add Q-decomposition to the DUSDi skill learner | 2D Gunner · Skill-discriminator accuracy | No clear effect | Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning (NeurIPS 2024 · 4 citations) | |
| Increase latent-space capacity | Ant and HalfCheetah · skill diversity | Helped | METRA: Scalable Unsupervised RL with Metric-Aware Abstraction (ICLR 2024 · 2 citations) | |
| Use more clusters than the default | AntMaze · RL performance | No clear effect | Subwords as Skills: Tokenization for Sparse-Reward Reinforcement Learning (NeurIPS 2024 · 2 citations) | |
| Use only one Do video (and one Don't video) instead of the default video set | Quadruped running · zero-shot run reward | No clear effect | Do's and Don'ts: Learning Desirable Skills with Instruction Videos (NeurIPS 2024 · 1 citation) | |
| Set λ to zero or a small value instead of using a larger MMD weight | Square-a maze · skill distinguishability | Hurt | Skill Disentanglement in Reproducing Kernel Hilbert Space (AAAI 2025 · 0 citations) | |
| add an RND bonus when pseudo-labeling offline data | representative state-based tasks · normalized return | Helped | Leveraging Skills from Unlabeled Prior Data for Efficient Online Exploration (ICML 2025 · 0 citations) | |
| reduce the ensemble value-function count while keeping the skill count unchanged | Quadruped downstream tasks · generalization performance | Hurt | Constrained Ensemble Exploration for Unsupervised Skill Discovery (ICML 2024 · 0 citations) | |
| Increase cross-embodiment pre-training timesteps | Image-based DMC downstream tasks · fine-tuned performance | Helped | PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learning (NeurIPS 2024 · 0 citations) | |
| Add dependency graphs to skill representations | Downstream tasks · success rate | Helped | SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions (NeurIPS 2024 · 0 citations) | |
| Combine PSD and METRA with independently trained encoders and summed intrinsic rewards, without mutual conditioning | MuJoCo locomotion · skill diversity | Hurt | Periodic Skill Discovery (NeurIPS 2025) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| CIC | Walker Flip | score | 631 – 715 | 2 papers, e.g. ICLR 2023 and NeurIPS 2022 |
| CIC | Walker Run | score | 486 – 535 | 2 papers, e.g. ICLR 2023 and NeurIPS 2022 |
What reviewers will expect
Datasets and baselines named in the 5 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Baselines
- DIAYN · 3 of 5 papers
- METRA · 2 of 5 papers
- DADS · 2 of 5 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 29 papers; from OpenAlex and the papers' affiliations).
- University of California, Berkeley · 3 papers
- Shanghai AI Laboratory · 3 papers
- Hong Kong University of Science and Technology · 3 papers
- Korea Advanced Institute of Science and Technology · 3 papers
- Google · 2 papers
- Northwestern Polytechnical University · 2 papers
- Seoul National University · 2 papers
- Sony · 2 papers
- University of Texas, Austin · 2 papers
- University of Texas at Austin · 2 papers
Use it
Download BibTeX of 37 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Novelty and impact-driven exploration (85 papers)
- Hierarchical skills and priors (84 papers)
- Reward shaping for sparse rewards (51 papers)
Elsewhere in the topic
- In-context and adaptive RL (202 papers)
- Scalable RL systems and environments (179 papers)
- Probabilistic and autonomous RL (177 papers)
- RL for combinatorial routing (158 papers)
- Learned state and action representations (148 papers)
- Dexterous manipulation transfer (141 papers)
- RL with tree search for discovery (121 papers)
- Inferring decision-making from behavior (93 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).