Research map / Continual learning / Online and replay-based
Understanding the Role of Training Regimes in Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace naive SGD training with dropout, learning-rate decay, and small-batch stable training | SGD · Permuted MNIST | average accuracy: 44.4 ($\pm$2.46) -> 80.1 ($\pm$0.51) | Helped |
| replace naive SGD training with dropout, learning-rate decay, and small-batch stable training | SGD · Permuted MNIST | average forgetting: 0.53 ($\pm$0.03) -> 0.09 ($\pm$0.01) | Helped |
| replace naive SGD training with dropout, learning-rate decay, and small-batch stable training | SGD · Rotated MNIST | average accuracy: 46.3 ($\pm$1.37) -> 70.8 ($\pm$0.78) | Helped |
| replace naive SGD training with dropout, learning-rate decay, and small-batch stable training | SGD · Rotated MNIST | average forgetting: 0.52 ($\pm$0.01) -> 0.10 ($\pm$0.02) | Helped |
| replace naive SGD training with dropout, learning-rate decay, and small-batch stable training | SGD · Split CIFAR-100 | average accuracy: 40.4 ($\pm$2.83) -> 59.9 ($\pm$1.81) | Helped |
| replace naive SGD training with dropout, learning-rate decay, and small-batch stable training | SGD · Split CIFAR-100 | average forgetting: 0.31 ($\pm$0.02) -> 0.08 ($\pm$0.01) | Helped |
| use stable training for both sequential tasks (ablation) | two-task SGD · Permuted MNIST | forgetting (F1): 19.37 $\pm$ (1.79) -> 1.61 $\pm$ (0.48) | Helped |
| use stable training for task 1, leaving task 2 plastic (ablation) | two-task SGD · Permuted MNIST | forgetting (F1): 19.37 $\pm$ (1.79) -> 4.77 $\pm$ (1.72) | Helped |
| use stable training for task 2, leaving task 1 plastic (ablation) | two-task SGD · Permuted MNIST | forgetting (F1): 19.37 $\pm$ (1.79) -> 12.45 $\pm$ (1.58) | Helped |
| add the stable training regime (ablation) | EWC · Rotated MNIST | average accuracy: 48.5 ($\pm$ 1.24) -> 61.4 ($\pm$ 1.15) | Helped |
| add the stable training regime (ablation) | A-GEM · Rotated MNIST | average accuracy: 55.3 ($\pm$ 1.47) -> 71.1 ($\pm$ 1.06) | Helped |
| add the stable training regime (ablation) | ER-Reservoir · Rotated MNIST | average accuracy: 69.2 ($\pm$ 1.10) -> 78.2 ($\pm$ 0.74) | Helped |
Closest papers by meaning
- STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.90)
- On the Diminishing Returns of Width for Continual Learning (ICML 2024 · similarity 0.90)
- Optimal Protocols for Continual Learning via Statistical Physics and Control Theory (ICLR 2025 · similarity 0.90)
- The Importance of Being Lazy: Scaling Limits of Continual Learning (ICML 2025 · similarity 0.90)
- When to Learn and When to Stop: Quitting at the Optimal Time (Student Abstract) (AAAI 2025 · similarity 0.90)
- Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.