Research map / Continual learning / Online and replay-based
Continual Deep Learning by Functional Regularisation of Memorable Past
Pingbo Pan, Siddharth Swaroop, Alexander Immer, Runa Eschenhagen, Richard Turner, Mohammad Emtiyaz Khan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FROMP instead of EWC | continual learning · Split CIFAR | forward transfer: 6.1±0.7% vs 0.17±0.9% | Improved |
| Use FROMP instead of VCL+coresets | continual learning · Split CIFAR | forward transfer: 6.1±0.7% vs 1.8±3.1% | Improved |
| Use FROMP instead of VCL+coresets | continual learning · Split CIFAR | backward transfer: -2.6±0.9% vs -9.2±1.8% | Improved |
| Use FROMP instead of EWC | continual learning · Split CIFAR | backward transfer: -2.6±0.9% vs -2.3±1.4% | No clear effect |
| Select memorable examples by the loss Hessian relevance instead of randomly (ablation) | FROMP · Split CIFAR with 10 memorable examples | average accuracy: 45% -> 70% | Improved |
| Select memorable examples by the loss Hessian relevance instead of randomly (ablation) | FROMP · Split MNIST | average accuracy | No clear effect |
| Replace the kernel matrix with the identity matrix (ablation) | FROMP · Permuted and Split MNIST | average accuracy | Hurt |
| Replace the kernel matrix with the identity matrix (ablation) | FROMP · Split CIFAR | average accuracy | No clear effect |
| Use random memorable examples instead of selected memorable examples (ablation) | FROMP · Split CIFAR with small memory | average accuracy: 70% vs 45% | Hurt |
| Use FROMP instead of separate per-task training | continual learning · Split CIFAR | average accuracy | Improved |
| Use FROMP instead of joint training | continual learning · Split CIFAR tasks 4–6 | accuracy | No clear effect |
| Use FROMP instead of weight-regularisation and FRCL methods | continual learning · Permuted MNIST and Split MNIST | average accuracy | Improved |
Closest papers by meaning
- Functional Regularisation for Continual Learning with Gaussian Processes (ICLR 2020 · similarity 0.91)
- A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning (TPAMI 2024 · similarity 0.87)
- A continual learning survey: Defying forgetting in classification tasks (TPAMI 2021 · similarity 0.87)
- Hyper-LifelongGAN: Scalable Lifelong Learning for Image Conditioned Generation (CVPR 2021 · similarity 0.87)
- Layerwise Optimization by Gradient Decomposition for Continual Learning (CVPR 2021 · similarity 0.86)
- Theory on Forgetting and Generalization of Continual Learning (ICML 2023 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
