Research map / Continual learning / Online and replay-based
Online Continual Learning under Extreme Memory Constraints
Enrico Fini, Stéphane Lathuilière, Enver Sangineto, Moin Nabi, Elisa Ricci
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dynamic gradient balancing of distillation and new-task gradients (ablation) | BLD · CIFAR10 (2 tasks) | average accuracy: 73.8 -> 78.0 | Improved |
| add dynamic gradient balancing of distillation and new-task gradients (ablation) | BLD · CIFAR10 (5 tasks) | average accuracy: 81.6 -> 86.2 | Improved |
| use joint training stage instead of alternating learning and distillation steps (ablation) | BLD · CIFAR10 (2 tasks) | average accuracy: 71.8 -> 78.0 | Improved |
| use joint training stage instead of alternating learning and distillation steps (ablation) | BLD · CIFAR10 (5 tasks) | average accuracy: 81.3 -> 86.2 | Improved |
| add two-stage distillation without balancing (ablation) | BLD · CIFAR10 (5 tasks) | average accuracy: 72.3 -> 81.6 | Improved |
| add batch-level distillation (BLD) to Finetune | Finetune · CIFAR10 (5 tasks) | average accuracy: 72.3 -> 86.2 | Improved |
| add batch-level distillation (BLD) to Finetune | Finetune · MNIST (5 tasks) | average accuracy: 80.6 -> 89.9 | Improved |
| add batch-level L2 parameter regularization to Finetune | Finetune · MNIST (5 tasks) | average accuracy: 80.6 -> 78.0 | Hurt |
| replace batch-level L2 with BLD | L2 · MNIST (2 tasks) | average accuracy: 95.7 -> 94.5 | Hurt |
| compare BLD with single-pass LwF that needs stored task data | Single-pass LwF · CIFAR10 (5 tasks) | average accuracy: 86.2 -> 86.2 | No clear effect |
| use BLD instead of offline multi-epoch LwF | Offline LwF · CIFAR10 (5 tasks) | average accuracy: 94.8 -> 86.2 | Hurt |
| iterate the joint training stage twice (ablation) | BLD · MC-OCL | accuracy | Improved |
Closest papers by meaning
- A Unified and General Framework for Continual Learning (ICLR 2024 · similarity 0.90)
- Improving Plasticity in Online Continual Learning via Collaborative Learning (CVPR 2024 · similarity 0.90)
- Mitigating Catastrophic Forgetting in Online Continual Learning by Modeling Previous Task Interrelations via Pareto Optimization (ICML 2024 · similarity 0.90)
- Online Task-Free Continual Learning via Dynamic Expansionable Memory Distribution (CVPR 2025 · similarity 0.90)
- Provable Contrastive Continual Learning (ICML 2024 · similarity 0.89)
- Advancing Prompt-Based Methods for Replay-Independent General Continual Learning (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
