Research map / Continual learning / Online and replay-based
Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UPGD, a utility-gated perturbed gradient descent that protects useful weights and perturbs less useful ones to address both catastrophic forgetting and loss of plasticity in streaming learning.
Problem
Deep networks in continual/streaming learning suffer from catastrophic forgetting of useful units and loss of plasticity (rigid, unuseful units), and most methods address only one issue or require task boundaries, replay buffers, or pretraining.
Approach
Define a weight utility via a second-order Taylor approximation of the loss change when a weight is zeroed (using linear-cost Hessian diagonals), then gate gradient updates by scaled utility so useful weights are protected while less useful weights receive gradient plus utility-scaled noise.
Key results
- UPGD-W continues to improve online accuracy on label-permuted EMNIST and CIFAR-10 streaming problems, outperforming methods that address only plasticity or only forgetting.
- On Input-Permuted MNIST, Shrink-and-Perturb and UPGD-W maintain performance while S-EWC decays.
- In PPO on MuJoCo, Adaptive UPGD continually improves performance while Adam suffers policy collapse (averaged over 40 runs).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use UPGD-W instead of methods without an explicit forgetting mechanism | continual learning with a streaming learner · Label-permuted CIFAR-10 | online accuracy | Helped |
| Use UPGD-W instead of methods that address loss of plasticity alone | continual learning with a streaming learner · Label-permuted EMNIST | online accuracy | Helped |
| Use AdaUPGD instead of Adam | PPO · MuJoCo environments | performance | Helped |
| Use UPGD-W instead of methods addressing loss of plasticity alone | continual learning with a streaming learner · Label-permuted mini-ImageNet | online accuracy | Helped |
| Use UPGD-W | continual learning with a streaming learner · Label-permuted mini-ImageNet | forgetting | Helped |
| Use UPGD-W | continual learning with a streaming learner · Label-permuted mini-ImageNet | loss of plasticity | Helped |
| Use UPGD or S&P instead of methods addressing catastrophic forgetting alone | continual learning with a streaming learner · Input-permuted MNIST | online accuracy | Helped |
| Use a utility-gated update rather than an update that gates perturbations only | UPGD · continual learning with a streaming learner | catastrophic forgetting | Helped |
| Use methods addressing catastrophic forgetting, including UPGD-W, rather than methods addressing loss of plasticity alone | continual learning with a streaming learner · Label-permuted EMNIST | online accuracy | Mixed |
Surprising findings
- Performance-based measures of plasticity correlate strongly with measured plasticity in the input-permuted setting, supporting performance as a plasticity proxy there.
- Methods that address only catastrophic forgetting (e.g., S-EWC) can decay in performance when loss of plasticity is also present, e.g., on Input-Permuted MNIST.
Limitations
- Reliance on first-order utility for fair comparisons; second-order variant only in appendix
- Convergence analysis only for stationary nonconvex problems with decreasing step size or increasing batch size
Experimental setup
Datasets: MNIST Input-Permuted MNIST Label-Permuted EMNIST Label-Permuted CIFAR-10 Label-Permuted mini-ImageNet MuJoCo
Benchmarks: Input-Permuted MNIST Label-Permuted EMNIST Label-Permuted CIFAR-10 Label-Permuted mini-ImageNet MuJoCo PPO environments
Metrics: online accuracy forgetting metric loss of plasticity metric Spearman rank correlation of utilities episodic return
Baselines: SGDW PGD Shrink and Perturb AdamW S-EWC S-SI S-MAS S-RWalk Non-protecting UPGD-W Adam Adam-Restarts
Builds on: Continual backprop (generate-and-test) Shrink and Perturb Perturbed gradient descent (Zhou et al. 2019) HesScale (Elsayed and Mahmood 2022) RWalk PPO
Closest papers by meaning
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning (NeurIPS 2025 · similarity 0.89)
- Improving Continual Learning Performance and Efficiency with Auxiliary Classifiers (ICML 2025 · similarity 0.89)
- Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification (ICLR 2025 · similarity 0.88)
- STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.88)
- Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning (NeurIPS 2026 · similarity 0.88)
- An Effective Dynamic Gradient Calibration Method for Continual Learning (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.