Research map / Continual learning / Online and replay-based
Overcoming Catastrophic Forgetting by Incremental Moment Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use mode-IMM instead of mean-IMM | IMM · Shuffled MNIST, tuned | accuracy: 95.02 ($\pm$ 0.42) -> 98.08 ($\pm$ 0.08) | Helped |
| Add L2-transfer to Mode-IMM (ablation) | Mode-IMM · Shuffled MNIST, tuned | accuracy: 98.08 ($\pm$ 0.08) -> 98.30 ($\pm$ 0.08) | Helped |
| Add drop-transfer to Mode-IMM (ablation) | Mode-IMM · Disjoint MNIST, tuned | accuracy: 92.02 ($\pm$ 0.73) -> 93.35 ($\pm$ 0.49) | Helped |
| Add L2-transfer and drop-transfer to Mode-IMM (ablation) | Mode-IMM · Disjoint MNIST, tuned | accuracy: 92.02 ($\pm$ 0.73) -> 94.12 ($\pm$ 0.27) | Helped |
| Use untuned rather than tuned mixing hyperparameters for L2-transfer + Mean-IMM (ablation) | L2-transfer + Mean-IMM · Disjoint MNIST | accuracy: 92.62 ($\pm$ 0.95) -> 78.34 ($\pm$ 1.82) | Hurt |
| Add transfer techniques to IMM (ablation) | IMM · Disjoint MNIST, tuned | accuracy | Helped |
| Initialize each new task from the previous task using weight-transfer (ablation) | IMM · Continual learning experiments | accuracy | Helped |
| Use the previous merged IMM model, rather than the previous task model, as the transfer reference (ablation) | IMM transfer techniques · Shuffled MNIST | accuracy | Hurt |
| Apply dropout regularization (ablation) | IMM and comparative models · Shuffled MNIST | accuracy | Helped |
| Use mode-IMM for Lifelog instead of mean-IMM | IMM · Lifelog, Activity | accuracy: 52.74 -> 54.07 | Helped |
| Use mode-IMM for Lifelog instead of mean-IMM | IMM · Lifelog, Location | accuracy: 77.60 -> 77.14 | Hurt |
Closest papers by meaning
- A comprehensive, application-oriented study of catastrophic forgetting in DNNs (ICLR 2019 · similarity 0.88)
- Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks (ICML 2025 · similarity 0.85)
- The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting — An Analytical Model (ICLR 2024 · similarity 0.85)
- Kolmogorov-Arnold Networks Still Catastrophically Forget but Differently from MLP (AAAI 2025 · similarity 0.85)
- Predicting the Susceptibility of Examples to Catastrophic Forgetting (ICML 2025 · similarity 0.85)
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.