Research map / Continual learning / Online and replay-based
When to Learn and When to Stop: Quitting at the Optimal Time (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Optimal Stopping (OS), which halts training on each new task when mean validation accuracy across all tasks stops rising, and combines it with Sleep Replay Consolidation to reduce catastrophic forgetting on split MNIST.
Problem
Catastrophic forgetting in continual learning: training each new task to maximal performance erases performance on previously learned tasks.
Approach
Stop training each new task as soon as the mean validation accuracy across all current and previous tasks stops increasing (reverting to prior parameters), trading new-task accuracy for retention; combined with sleep-phase replay (SRC) via spiking conversion and Hebbian plasticity.
Key results
- Baseline continual training ends with mean accuracy around 20%, with previous tasks dropping to zero accuracy.
- Optimal Stopping raises mean accuracy from about 20% to about 35%; OS+SRC reaches about 45%, over twice baseline.
- The standard SRC implementation reached about 60% mean accuracy, above the 48% originally reported, and OS+SRC does not exceed it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use optimal stopping instead of fixed-epoch training | 4-layer fully connected ANN · MNIST (5 tasks) | mean accuracy: about 20% to 35% | Helped |
| Add Sleep Replay Consolidation to optimal stopping | OS · MNIST (5 tasks) | mean accuracy: 35% to around 45% | Helped |
| Use optimal stopping with SRC rather than fixed-epoch training with SRC | Baseline + SRC · MNIST (5 tasks) | mean accuracy: ∼60% to around 45% | Hurt |
| Add SRC to fixed-epoch training | Baseline · MNIST (5 tasks) | mean accuracy: around 20% to ∼60% | Helped |
Surprising findings
- Standard SRC alone outperforms OS+SRC in mean accuracy, though OS+SRC shows smoother performance across training.
Limitations
- Results are limited to a small fully connected network on split MNIST with five tasks.
- OS+SRC does not exceed standard SRC performance in the reported experiments.
- Hyperparameters are deferred to a supplement and not described in the main text.
Experimental setup
Datasets: MNIST
Benchmarks: MNIST split task (5 tasks of two digits each)
Metrics: mean accuracy across tasks per-task accuracy
Baselines: Baseline continual training (fixed epochs) Baseline + SRC SRC (Tadros et al. 2022)
Builds on: Sleep Replay Consolidation (SRC) early stopping
Closest papers by meaning
- Do Your Best and Get Enough Rest for Continual Learning (CVPR 2025 · similarity 0.90)
- Optimal Protocols for Continual Learning via Statistical Physics and Control Theory (ICLR 2025 · similarity 0.89)
- Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective (ICML 2025 · similarity 0.89)
- Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning (ICML 2024 · similarity 0.89)
- STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.89)
- Predicting the Susceptibility of Examples to Catastrophic Forgetting (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.