Research map / Continual learning / Online and replay-based
Plasticity-preserving continual reinforcement learning
Keeps policies adaptable using architectural design, policy subspaces and composition across tasks.
43 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Online and replay-based, keyword cluster “weight · transfer · growing”
State History Tried Baselines Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
8 papers in 2023–24 and 27 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 4.9% to 13.7%.
Open questions the data points to
- Papers report EWC on CW20 (forward transfer) anywhere from -0.17 to -0.15: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Continual Planning in Golog (AAAI 2016 · 16 citations)
- 2018 · Modular Continual Learning in a Unified Visual Environment (ICLR 2018 · 0 citations)
- 2019 · Policy Consolidation for Continual Reinforcement Learning (ICML 2019 · 10 citations)
- 2021 · Continual World: A Robotic Benchmark For Continual Reinforcement Learning (NeurIPS 2021 · 24 citations)
- 2022 · Towards Better Plasticity-Stability Trade-Off in Incremental Learning: A Simple Linear Connector (CVPR 2022 · 57 citations)
- 2023 · Building a Subspace of Policies for Scalable Continual Learning (ICLR 2023 · 3 citations)
- 2024 · Parseval Regularization for Continual Reinforcement Learning (NeurIPS 2024 · 2 citations)
- 2025 · Knowledge Retention in Continual Model-Based Reinforcement Learning (ICML 2025 · 0 citations)
- 2026 · Balancing Plasticity and Stability with Fast and Slow Successor Features (ICML 2026 · 0 citations)
Earliest papers here
- Continual Planning in Golog (AAAI 2016 · 16 citations)
- Modular Continual Learning in a Unified Visual Environment (ICLR 2018 · 0 citations)
- Policy Consolidation for Continual Reinforcement Learning (ICML 2019 · 10 citations)
What has been tried
21 papers here report 227 measured changes (a component added, removed or swapped, compared with a baseline): helped 63%, no clear effect 10%, hurt 23%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add diagonal layers to Parseval regularization | Metaworld sequences · performance | Helped | Parseval Regularization for Continual Reinforcement Learning (NeurIPS 2024 · 2 citations) | |
| Greedy-selection search instead of open-ended exploration for memory design learning | ALFWorld, GPT-5-nano and GPT-5-mini testing · success rate | 11.9 vs 12.4 (nano); 77.1 vs 87.1 (mini) | Hurt | Learning to Continually Learn via Meta-learning Agentic Memory Designs (NeurIPS 2026 · 0 citations) |
| Remove Synthetic Experience Rehearsal | Cheetah and MiniGrid transfer tasks · average reward | Hurt | Knowledge Retention in Continual Model-Based Reinforcement Learning (ICML 2025 · 0 citations) | |
| Use the orthogonal churn-reduction gradient component only | ProcGen Starpilot and Chaser · episode return | Helped | Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churn (ICML 2025 · 0 citations) | |
| replace vector quantization with an auto-encoder | D4RL continual learning · normalized score | Hurt | Tackling Continual Offline RL through Selective Weights Activation on Aligned Spaces (NeurIPS 2025 · 0 citations) | |
| add adaptive weighting of historical knowledge vectors | Freeway · PERF. | 0.7532 → 0.7821 | Helped | Continual Knowledge Adaptation for Reinforcement Learning (NeurIPS 2025 · 0 citations) |
| add feature distillation (KD) | HM3D and MP3D · average SR | 22% and 16% | Helped | C-NAV: Towards Self-Evolving Continual Object Navigation in Open World (NeurIPS 2025 · 0 citations) |
| Use context-aware composition rather than fixed composition | DMC Walker continual policy shift · performance | Helped | Skill Expansion and Composition in Parameter Space (ICLR 2025 · 0 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| EWC | CW20 | forward transfer | -0.17 – -0.15 | 2 papers, e.g. NeurIPS 2021 and NeurIPS 2022 |
| PackNet | CW20 | forward transfer | 0.18 – 0.19 | 2 papers, e.g. NeurIPS 2022 and NeurIPS 2021 |
| Fine-tuning | CW20 | forward transfer | 0.19 – 0.2 | 2 papers, e.g. NeurIPS 2022 and NeurIPS 2021 |
What reviewers will expect
Datasets and baselines named in the 12 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- D4RL · 2 of 12 papers
Baselines
- EWC · 2 of 12 papers
- PackNet · 2 of 12 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 35 papers; from OpenAlex and the papers' affiliations).
- University of British Columbia · 2 papers
- McGill University · 2 papers
- Sun Yat-sen University · 2 papers
Use it
Download BibTeX of 32 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Theoretical analysis of forgetting (141 papers)
- Unified and unsupervised continual frameworks (120 papers)
- Gradient projection and plasticity preservation (80 papers)
- Dynamic expansion and isolation (70 papers)
Elsewhere in the topic
- Memory-based and Bayesian online learning (56 papers)
- Plug-and-play online training enhancements (39 papers)
- Replay buffer construction and update (33 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).