Research map / Continual learning / Online and replay-based
Dynamic expansion and isolation
Grows or partitions networks and generators to keep task knowledge separate.
70 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Online and replay-based, keyword cluster “weight · transfer · growing”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
26 papers in 2023–24 and 26 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 16.0% to 13.2%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · ForgetMeNot: Memory-Aware Forensic Facial Sketch Matching (CVPR 2016 · 69 citations)
- 2018 · Forgetting and Unfolding for Existential Rules (AAAI 2018 · 6 citations)
- 2019 · Incremental Object Learning From Contiguous Views (CVPR 2019 · 35 citations)
- 2020 · iTAML: An Incremental Task-Agnostic Meta-learning Approach (CVPR 2020 · 139 citations)
- 2021 · Image De-Raining via Continual Learning (CVPR 2021 · 50 citations)
- 2022 · Continual Stereo Matching of Continuous Driving Scenes With Growing Architecture (CVPR 2022 · 13 citations)
- 2023 · Isolation and Impartial Aggregation: A Paradigm of Incremental Learning without Interference (AAAI 2023 · 51 citations)
- 2024 · Towards Continual Knowledge Graph Embedding via Incremental Distillation (AAAI 2024 · 42 citations)
- 2025 · One-for-More: Continual Diffusion Model for Anomaly Detection (CVPR 2025 · 24 citations)
- 2026 · Unifying Locality of KANs and Feature Drift Compensation Projection for Data-Free Replay Based Continual Face Forgery Detection (AAAI 2026 · 1 citation)
Earliest papers here
- ForgetMeNot: Memory-Aware Forensic Facial Sketch Matching (CVPR 2016 · 69 citations)
- Forgetting and Unfolding for Existential Rules (AAAI 2018 · 6 citations)
- Dependence in Propositional Logic: Formula-Formula Dependence and Formula Forgetting – Application to Belief Update and Conservative Extension (AAAI 2018 · 1 citation)
What has been tried
57 papers here report 570 measured changes (a component added, removed or swapped, compared with a baseline): helped 71%, no clear effect 7%, hurt 18%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add incremental distillation | ENTITY, RELATION, FACT, HYBRID, GraphEqual, GraphHigher, and GraphLower · MRR | 0.2\%-6.5\% | Helped | Towards Continual Knowledge Graph Embedding via Incremental Distillation (AAAI 2024 · 42 citations) |
| Increase client cache memory from M = 1000 to M = 3000 | Digit10 · accuracy | 82.30±0.16 -> 86.32±0.04 | Helped | Towards Efficient Replay in Federated Incremental Learning (CVPR 2024 · 34 citations) |
| Add self-learned gradient-modified radian (LRR) | ASVspoof2019LA, S→ASVspoof2015 · EER (%) | 2.448 -> 0.438 | Helped | What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection (AAAI 2024 · 30 citations) |
| Add the anomaly-masked network (AMN) | MVTec 10–1 × 5 with 5 steps · image A-AUROC | 94.3 -> 94.9 | Helped | One-for-More: Continual Diffusion Model for Anomaly Detection (CVPR 2025 · 24 citations) |
| replace vanilla influence-based replay with one-step influence estimation | Gowalla and Foursquare · Recall@20 | 0.0591 -> 0.0621; 0.0847 -> 0.0876; 0.0913 -> 0.0944; 0.1163 -> 0.1249 | Helped | Influential Exemplar Replay for Incremental Learning in Recommender Systems (AAAI 2024 · 23 citations) |
| add incremental concept discovery to the initial base concept bank | LAD and CUB (Mean) · accuracy | 0.5812 -> 0.7009 | Helped | Incremental Residual Concept Bottleneck Models (CVPR 2024 · 21 citations) |
| Add incremental learning | LIVE → CSIQ · PLCC | 0.957 -> 0.963 | Helped | Semi-Supervised Blind Image Quality Assessment through Knowledge Distillation and Incremental Learning (AAAI 2024 · 14 citations) |
| Add DAM and IGAM | Rain100H (Setting 1) · PSNR | 13.47 -> 28.91 | Helped | Continuous Adverse Weather Removal via Degradation-Aware Distillation (CVPR 2025 · 13 citations) |
| add Incremental Decision Alignment (IDA) | Protocol 1 T4 average · AUC | 0.8352 -> 0.9315 | Helped | Stacking Brick by Brick: Aligned Feature Isolation for Incremental Face Forgery Detection (CVPR 2025 · 10 citations) |
| Remove TSOS while keeping Lg′ | Split MNIST · reconstruction FID | 30.57 -> 33.84 | Hurt | Task-Free Continual Generation and Representation Learning via Dynamic Expansionable Memory Cluster (AAAI 2024 · 10 citations) |
| Add knowledge replay using assistant-client features | CIFAR-10, ResNet18, non-IID and IID settings · test accuracy | Helped | MultiSFL: Towards Accurate Split Federated Learning via Multi-Model Aggregation and Knowledge Replay (AAAI 2025 · 9 citations) | |
| Add Ebbinghaus Forgetting Mechanism (EFM) | EVDA Avg · EER (%) | 6.80 -> 6.34 | Helped | Region-Based Optimization in Continual Learning for Audio Deepfake Detection (AAAI 2025 · 7 citations) |
What reviewers will expect
Datasets and baselines named in the 18 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 4 of 18 papers
- CIFAR-100 · 3 of 18 papers
- MNIST · 2 of 18 papers
- SVHN · 2 of 18 papers
- Mini-ImageNet · 2 of 18 papers
- TinyImageNet · 2 of 18 papers
Baselines
- LwF · 6 of 18 papers
- EWC · 5 of 18 papers
- GEM · 3 of 18 papers
- Joint · 2 of 18 papers
- Finetune · 2 of 18 papers
- MAS · 2 of 18 papers
- DER · 2 of 18 papers
- PackNet · 2 of 18 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 52 papers; from OpenAlex and the papers' affiliations).
- Chinese Academy of Sciences · 7 papers
- Georgia Institute of Technology · 3 papers
- University of Science and Technology of China · 3 papers
- Peng Cheng Laboratory · 3 papers
- Tencent · 3 papers
- University of Electronic Science and Technology of China · 3 papers
- Harbin Institute of Technology · 2 papers
- Singapore Management University · 2 papers
- University of York · 2 papers
- Southeast University · 2 papers
Use it
Download BibTeX of 33 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Theoretical analysis of forgetting (141 papers)
- Unified and unsupervised continual frameworks (120 papers)
- Gradient projection and plasticity preservation (80 papers)
- Plasticity-preserving continual reinforcement learning (43 papers)
Elsewhere in the topic
- Memory-based and Bayesian online learning (56 papers)
- Plug-and-play online training enhancements (39 papers)
- Replay buffer construction and update (33 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).