Research map / Continual learning / Online and replay-based
Replay buffer construction and update
Improves what is stored and replayed, and how, to reduce representation drift online.
33 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2018–2026) · an approach in Online and replay-based, keyword cluster “online continual · replay · experience”
State History Tried Baselines Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
12 papers in 2023–24 and 5 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 7.4% to 2.5%.
Open questions the data points to
- Papers report MER on Rotated MNIST (Accuracy) anywhere from 66 to 83.56: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2018 · Selective Experience Replay for Lifelong Learning (AAAI 2018 · 355 citations)
- 2019 · Experience Replay for Continual Learning (NeurIPS 2019 · 368 citations)
- 2021 · Rainbow Memory: Continual Learning With a Memory of Diverse Samples (CVPR 2021 · 347 citations)
- 2022 · New Insights on Reducing Abrupt Representation Change in Online Continual Learning (ICLR 2022 · 43 citations)
- 2023 · Regularizing Second-Order Influences for Continual Learning (CVPR 2023 · 24 citations)
- 2024 · Summarizing Stream Data for Memory-Constrained Online Continual Learning (AAAI 2024 · 16 citations)
- 2025 · Gradient-Guided Epsilon Constraint Method for Online Continual Learning (NeurIPS 2025 · 0 citations)
- 2026 · Trust Region Continual Learning as an Implicit Meta-Learner (NeurIPS 2026 · 0 citations)
Earliest papers here
- Selective Experience Replay for Lifelong Learning (AAAI 2018 · 355 citations)
- Memory Replay GANs: Learning to Generate New Categories without Forgetting (NeurIPS 2018 · 212 citations)
- Experience Replay for Continual Learning (NeurIPS 2019 · 368 citations)
What has been tried
27 papers here report 312 measured changes (a component added, removed or swapped, compared with a baseline): helped 72%, no clear effect 9%, hurt 15%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add dynamic memory to SCR | CIFAR-100, K=100/500/1000 · average end accuracy | 9.0 -> 10.7; 20.6 -> 21.3; 26.6 -> 27.2 | Helped | Summarizing Stream Data for Memory-Constrained Online Continual Learning (AAAI 2024 · 16 citations) |
| Modulate all five feature points instead of only the first three | Split Mini-ImageNet Class-IL, buffer 2000 · accuracy | 12.97±2.62 -> 22.92±2.25 | Helped | Saliency-driven Experience Replay for Continual Learning (NeurIPS 2024 · 9 citations) |
| add isotropic replay | CIFAR10, PGD→FGSM→None · Robustness (%) | Helped | Defense without Forgetting: Continual Adversarial Defense with Anisotropic & Isotropic Pseudo Replay (CVPR 2024 · 7 citations) | |
| Use T=5 and p=0.05 for preconditioner updating | Split-CIFAR100, memory 2000 · accuracy | 0.2937 { ± 0.0054} -> 0.3307 { ± 0.0038} | Helped | Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning (ICML 2024 · 0 citations) |
| Add the hyper-gradient implementation | Split CIFAR-10 (|M|=1k) · AAA | 39.03 -> 66.23 | Helped | Mitigating Catastrophic Forgetting in Online Continual Learning by Modeling Previous Task Interrelations via Pareto Optimization (ICML 2024) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| MER | Rotated MNIST | Accuracy | 66 – 83.56 | 2 papers, e.g. AAAI 2021 and ICLR 2021 |
What reviewers will expect
Datasets and baselines named in the 8 papers of this approach from 2024–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-100 · 2 of 8 papers
- Mini-ImageNet · 2 of 8 papers
- MNIST · 2 of 8 papers
- CIFAR-10 · 2 of 8 papers
- Split-CIFAR100 · 2 of 8 papers
- Split-TinyImageNet · 2 of 8 papers
Baselines
- EWC · 4 of 8 papers
- AGEM · 3 of 8 papers
- ER · 3 of 8 papers
- DER · 3 of 8 papers
- LwF · 2 of 8 papers
- MIR · 2 of 8 papers
- Experience Replay · 2 of 8 papers
- ER-ACE · 2 of 8 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 17 papers; from OpenAlex and the papers' affiliations).
- Alibaba Group · 2 papers
- Ant Group · 2 papers
- City University of Hong Kong · 2 papers
- University of Science and Technology of China · 2 papers
Use it
Download BibTeX of 27 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Memory-based and Bayesian online learning (56 papers)
- Plug-and-play online training enhancements (39 papers)
Elsewhere in the topic
- Theoretical analysis of forgetting (141 papers)
- Unified and unsupervised continual frameworks (120 papers)
- Gradient projection and plasticity preservation (80 papers)
- Dynamic expansion and isolation (70 papers)
- Plasticity-preserving continual reinforcement learning (43 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).