Research map / Reinforcement learning / Visual and representation RL
In-context and adaptive RL
Enable fast adaptation in RL via in-context learning, memory, and domain transfer.
202 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Visual and representation RL, keyword cluster “search · decision · systems”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
50 papers in 2023–24 and 72 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 8.7% to 10.3%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Near Optimal Behavior via Approximate State Abstraction (ICML 2016 · 111 citations)
- 2017 · FeUdal Networks for Hierarchical Reinforcement Learning (ICML 2017 · 245 citations)
- 2018 · Using Reward Machines for High-Level Task Specification and Decomposition in Reinforcement Learning (ICML 2018 · 128 citations)
- 2019 · Hierarchical Reinforcement Learning for Course Recommendation in MOOCs (AAAI 2019 · 191 citations)
- 2020 · Tree-Structured Policy Based Progressive Reinforcement Learning for Temporally Language Grounding in Video (AAAI 2020 · 119 citations)
- 2021 · Hierarchical Reinforcement Learning for Integrated Recommendation (AAAI 2021 · 77 citations)
- 2022 · The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models (ICLR 2022 · 21 citations)
- 2023 · A Toolkit for Reliable Benchmarking and Research in Multi-Objective Reinforcement Learning (NeurIPS 2023 · 20 citations)
- 2024 · Assume-Guarantee Reinforcement Learning (AAAI 2024 · 5 citations)
- 2025 · Autonomous Option Invention for Continual Hierarchical Reinforcement Learning and Planning (AAAI 2025 · 3 citations)
- 2026 · Cerebellar-Inspired Residual Control for Fault Recovery: From Inference-Time Adaptation to Structural Consolidation (ICML 2026 · 0 citations)
Earliest papers here
- Near Optimal Behavior via Approximate State Abstraction (ICML 2016 · 111 citations)
- Collective Noise Contrastive Estimation for Policy Transfer Learning (AAAI 2016 · 17 citations)
- FeUdal Networks for Hierarchical Reinforcement Learning (ICML 2017 · 245 citations)
What has been tried
149 papers here report 1,440 measured changes (a component added, removed or swapped, compared with a baseline): helped 65%, no clear effect 10%, hurt 22%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add value prompts | Hard diverse datasets · evaluation performance | Helped | Rethinking Decision Transformer via Hierarchical Reinforcement Learning (ICML 2024 · 4 citations) | |
| replace SDN with MPPN while using DQN | JPLEG-5 test · Perfect | 5.1 -> 9.8; +5.1% | Helped | ERL-MPP: Evolutionary Reinforcement Learning with Multi-head Puzzle Perception for Solving Large-scale Jigsaw Puzzles of Eroded Gaps (AAAI 2025 · 3 citations) |
| add pre-verification to generated latent-reward code | CN (6 agents) · Execution rate | 0% -> 100% | Helped | Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning (AAAI 2025 · 3 citations) |
| Increase the expert count from 3 to 8 | Walker2d-v4 · avg. ER | 3920.91 ± 14.42 -> 4224.29 ± 25.96 | Helped | SMoSE: Sparse Mixture of Shallow Experts for Interpretable Reinforcement Learning in Continuous Control Tasks (AAAI 2025 · 3 citations) |
| Increase demonstration size | MORL tasks · average utility | Helped | An Offline Adaptation Framework for Constrained Multi-Objective Reinforcement Learning (NeurIPS 2024 · 3 citations) | |
| Increase particle count from 100 to 4000 | STE · success rate | 88.21% -> 95.27% | Helped | Autonomous Goal Detection and Cessation in Reinforcement Learning: A Case Study on Source Term Estimation (AAAI 2025 · 2 citations) |
| add task-wise MoE | Cheetah-Vel (Mixed) · test return | -76.1 -> -68.9 | Helped | Mixture-of-Experts Meets In-Context Reinforcement Learning (NeurIPS 2025 · 2 citations) |
| vary the number of cut points and use an intermediate value | MuJoCo: InvertedPendulum, Hopper, Swimmer, Walker2d · episodic return | Helped | Episodic Return Decomposition by Difference of Implicitly Assigned Sub-trajectory Reward (AAAI 2024 · 2 citations) | |
| Restrict alerts to days above a county-specific QHI threshold | Evaluation years 2007, 2011, 2015 · Median difference in average return vs NWS | DQN: -0.123 -> 0.03; QR-DQN: -0.117 -> 0.035; TRPO: -0.065 -> 0.038; A2C: -0.063 -> 0.042 | Helped | Optimizing Heat Alert Issuance with Reinforcement Learning (AAAI 2025 · 1 citation) |
| Add the solver-step constraint and physics-motivated reward | Noise-free Λ system · mean fidelity | Helped | Reinforcement Learning for Quantum Control under Physical Constraints (ICML 2025 · 1 citation) | |
| Prioritize environments with the highest estimated error (WAKER-M) rather than the highest error-reduction rate (WAKER-R) | Robustness evaluation across domains · CVaR0.1 performance | Helped | Reward-Free Curricula for Training Robust World Models (ICLR 2024 · 1 citation) | |
| Add prompt decomposition | Cheetah-vel, MW ML45, MW MT50 · average return | -163.05 -> -139.88; 108.01 -> 347.21; 709.81 -> 1559.94 | Helped | Decomposed Prompt Decision Transformer for Efficient Unseen Task Generalization (NeurIPS 2024 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 44 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- D4RL Adroit · 2 of 44 papers
- D4RL AntMaze · 2 of 44 papers
- Cheetah-vel · 2 of 44 papers
- MuJoCo HalfCheetah · 2 of 44 papers
- ALFWorld · 2 of 44 papers
- MiniGrid · 2 of 44 papers
- Point-Robot · 2 of 44 papers
Baselines
- PPO · 5 of 44 papers
- DQN · 4 of 44 papers
- Random · 3 of 44 papers
- Greedy Search · 2 of 44 papers
- Decision Transformer · 2 of 44 papers
- CQL · 2 of 44 papers
- IQL · 2 of 44 papers
- ReAct · 2 of 44 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 122 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 7 papers
- University of Oxford · 6 papers
- Google DeepMind · 5 papers
- Massachusetts Institute of Technology · 5 papers
- Harvard University · 4 papers
- The Chinese University of Hong Kong · 4 papers
- Microsoft · 4 papers
- Korea Advanced Institute of Science and Technology · 4 papers
- University of California, Los Angeles · 3 papers
- Nanjing University · 3 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Scalable RL systems and environments (179 papers)
- Probabilistic and autonomous RL (177 papers)
- RL for combinatorial routing (158 papers)
- RL with tree search for discovery (121 papers)
- Inferring decision-making from behavior (93 papers)
- Curriculum and environment design (81 papers)
Elsewhere in the topic
- Learned state and action representations (148 papers)
- Dexterous manipulation transfer (141 papers)
- Skill learning and diffusion policies (86 papers)
- Robust visual encoders for RL (86 papers)
- Novelty and impact-driven exploration (85 papers)
- Hierarchical skills and priors (84 papers)
- Latent-space world models (83 papers)
- Goal-conditioned policy learning (66 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).