Research map / Reinforcement learning / Visual and representation RL
Task structure and curricula for meta-RL
Exploit subtask structure, curricula, and sampling choices to improve meta-RL adaptation.
44 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Visual and representation RL, keyword cluster “meta · adaptation · context”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
15 papers in 2023–24 and 12 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 2.6% to 1.7%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning (ICML 2017 · 109 citations)
- 2018 · META LEARNING SHARED HIERARCHIES (ICLR 2018 · 111 citations)
- 2019 · Unsupervised Curricula for Visual Meta-Reinforcement Learning (NeurIPS 2019 · 26 citations)
- 2020 · MetaLight: Value-Based Meta-Reinforcement Learning for Traffic Signal Control (AAAI 2020 · 169 citations)
- 2021 · Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agents (NeurIPS 2021 · 6 citations)
- 2022 · FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning (NeurIPS 2022 · 48 citations)
- 2023 · Human-Timescale Adaptation in an Open-Ended Task Space (ICML 2023 · 20 citations)
- 2024 · Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitations (AAAI 2024 · 12 citations)
- 2025 · ConfigX: Modular Configuration for Evolutionary Algorithms via Multitask Reinforcement Learning (AAAI 2025 · 4 citations)
- 2026 · Out-of-Distribution Generalization with a SPARC: Racing 100 Unseen Vehicles with a Single Policy (AAAI 2026 · 1 citation)
Earliest papers here
- Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning (ICML 2017 · 109 citations)
- META LEARNING SHARED HIERARCHIES (ICLR 2018 · 111 citations)
- The Importance of Sampling inMeta-Reinforcement Learning (NeurIPS 2018 · 11 citations)
What has been tried
36 papers here report 335 measured changes (a component added, removed or swapped, compared with a baseline): helped 64%, no clear effect 13%, hurt 18%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| use reconstruction training for TAE instead of a contrastive objective | Ant-Dir Test (one-shot) · return | 376.37±110.18 -> 464.11±95.74 | Helped | Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitations (AAAI 2024 · 12 citations) |
| replace MSA blocks with an MLP layer | T_test,in, T_test,out^(1), and T_test,out^(2) · normalized accumulated optimization improvement | 9.81E-01 -> 9.70E-01; 9.86E-01 -> 9.80E-01; 9.22E-01 -> 9.16E-01 | Hurt | ConfigX: Modular Configuration for Evolutionary Algorithms via Multitask Reinforcement Learning (AAAI 2025 · 4 citations) |
| Use bf16 precision during training | PPO training throughput · steps per second | 1.2M -> 1.5M | Helped | XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX (NeurIPS 2024 · 2 citations) |
| Increase training task count from limited coverage to 10K or more | unseen AnyMDP tasks · validation loss / generalization | Helped | Towards Large-Scale In-Context Reinforcement Learning by Meta-Training in Randomized Worlds (NeurIPS 2025 · 1 citation) | |
| Add virtual-task training | Cheetah-Vel-OOD and Walker-Mass-OOD · average return | Helped | Task-Aware Virtual Training: Enhancing Generalization in Meta-Reinforcement Learning for Out-of-Distribution Tasks (ICML 2025 · 0 citations) | |
| Add session consistency regularization | offline Gridworld, Reacher, HC-Dir, HC-Vel, HC-Dir+Vel, ScratchItch · average episode return | 38.2 ± 1.2 vs 41.8 ± 0.6; -33.2 ± 2.7 vs -15.5 ± 3.2; -206.0 ± 5.6 vs -154.0 ± 8.6; -212.0 ± 6.4 vs -156.0 ± 4.8; -120.0 | Helped | DynaMITE-RL: A Dynamic Model for Improved Temporal Meta-Reinforcement Learning (NeurIPS 2024 · 0 citations) |
| Add VAE KL regularization | Two-arm Bernoulli bandit · state dissimilarity | 2.44±0.38; 0.21±0.04 | Helped | Predictive Coding Enhances Meta-RL To Achieve Interpretable Bayes-Optimal Belief Representation Under Partial Observability (NeurIPS 2025) |
| Use a dynamic window that rolls forward | Gridworld 6x6 OOD · average returns | Helped | Meta-Reinforcement Learning Robust to Distributional Shift Via Performing Lifelong In-Context Learning (ICML 2024) |
What reviewers will expect
Datasets and baselines named in the 8 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- HPO-B · 2 of 8 papers
Baselines
- RL$^2$ · 2 of 8 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 27 papers; from OpenAlex and the papers' affiliations).
- South China University of Technology · 3 papers
- Massachusetts Institute of Technology · 2 papers
- Google · 2 papers
- Google DeepMind · 2 papers
- University of Oxford · 2 papers
- University of Science and Technology of China · 2 papers
Use it
Download BibTeX of 34 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Task-inference meta-RL (57 papers)
Elsewhere in the topic
- In-context and adaptive RL (202 papers)
- Scalable RL systems and environments (179 papers)
- Probabilistic and autonomous RL (177 papers)
- RL for combinatorial routing (158 papers)
- Learned state and action representations (148 papers)
- Dexterous manipulation transfer (141 papers)
- RL with tree search for discovery (121 papers)
- Inferring decision-making from behavior (93 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).