Research map / Continual learning / Online and replay-based
A comprehensive, application-oriented study of catastrophic forgetting in DNNs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Fisher-weight regularization with EWC | DNNs trained sequentially · D9-1 tasks | incremental learning quality: EMNIST: .88/.09 -> .92/.51; MADBase: .89/.10 -> .99/.70 | Mixed |
| Apply Fisher-weight regularization with EWC | DNNs trained sequentially · D5-5 tasks | incremental learning quality: CIFAR10: .32/.20; EMNIST: .36/.08 | No clear effect |
| Use the “best” rather than the “last” evaluation criterion (ablation) | EWC on D9-1 tasks · EMNIST D9-1 | incremental learning quality: .92/.51 | Mixed |
| Use permutation-based DP10-10 tasks instead of class-incremental tasks | FC, CONV, D-FC, D-CONV, LWTA, EWC, and IMM · DP10-10 across tested datasets | catastrophic forgetting: MNIST FC: .99/.99; MNIST EWC: 1.0/.98 | Helped |
| Use Dropout with fully connected networks (ablation) | FC · D5-5 tasks | incremental learning quality: CIFAR10 FC: .30/.28; D-FC: .26/.23 | No clear effect |
| Use Dropout with convolutional networks (ablation) | CONV · D5-5 tasks | incremental learning quality: CIFAR10 CONV: .31/.10; D-CONV: .30/.18 | Mixed |
| Use LWTA activations | Fully connected DNNs · D5-5 tasks | incremental learning quality: MNIST LWTA: .10/.09; SVHN LWTA: .40/.20 | Mixed |
| Merge task-specific weights with IMM | DNNs trained sequentially · D5-5 tasks | incremental learning quality: CIFAR10 wtIMM mode: .31; Devanagari wtIMM mode: .73 | Helped |
| Use mean-IMM instead of mode-IMM (ablation) | wtIMM · CIFAR10 D5-5 | incremental learning quality: .31 -> .30 | No clear effect |
| Use a small second task (D9-1) rather than an equally sized task (D5-5) | EWC · EMNIST | incremental learning quality: D5-5: .36/.08; D9-1: .92/.51 | Helped |
| Select a fixed architecture using knowledge of future tasks | IMM · MNIST | incremental learning quality | Mixed |
| Tune IMM’s balancing parameter using cross-validation | IMM · Sequential classification tasks | application feasibility | Hurt |
Closest papers by meaning
- Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks (ICML 2025 · similarity 0.93)
- On the Diminishing Returns of Width for Continual Learning (ICML 2024 · similarity 0.88)
- Predicting the Susceptibility of Examples to Catastrophic Forgetting (ICML 2025 · similarity 0.88)
- Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning (ICLR 2025 · similarity 0.88)
- The Importance of Being Lazy: Scaling Limits of Continual Learning (ICML 2025 · similarity 0.88)
- Kolmogorov-Arnold Networks Still Catastrophically Forget but Differently from MLP (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.