Research map / Continual learning / Online and replay-based
Optimal Protocols for Continual Learning via Statistical Physics and Control Theory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uses Pontryagin's maximum principle on exact statistical-physics ODEs for online SGD in a two-layer teacher-student continual learning model to derive optimal replay (task-selection) and learning-rate protocols, and validates a heuristic "pseudo-optimal" focus-then-revision strategy on Fashion-MNIST.
Problem
Catastrophic forgetting in sequential/multi-task learning is usually mitigated by heuristic replay schedules with no guarantee of optimality; the paper asks what task-selection and learning-rate protocols minimise final loss across tasks.
Approach
Combine low-dimensional ODEs for online SGD order parameters (overlaps and readouts) in high-dimensional teacher-student models with optimal control (Pontryagin's maximum principle, forward-backward iteration) to solve for the optimal task-selection and learning-rate schedules, then distill an interpretable focus-then-revision heuristic.
Key results
- Optimal replay in the synthetic teacher-student setting yields a focus phase on the new task followed by interleaved revision, outperforming no-replay and interleaved baselines in final loss.
- With optimal task selection, forgetting is minimal at intermediate task similarity, reversing the non-monotonic no-replay trend reported in prior work.
- A closed-form convergence timescale for the readout layer is derived: alpha_conv = 3pi / (eta(pi - 6 arcsin(gamma/2))), diverging as gamma approaches 1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use optimal replay instead of no replay | continual learning on synthetic teacher-student tasks · synthetic continual learning | average loss | Helped |
| use interleaved replay instead of no replay | continual learning on synthetic teacher-student tasks · synthetic continual learning | average loss | Helped |
| randomize task ordering while keeping the optimal replay fraction (ablation) | optimal replay strategy · synthetic continual learning | average loss | Hurt |
| retain the focus phase and randomize the revision phase (ablation) | optimal replay strategy · synthetic continual learning | average loss | No clear effect |
| add an initial focus phase on the new task before interleaved replay (ablation) | interleaved replay strategy · synthetic continual learning | average loss | Helped |
| jointly optimize task selection and learning rate | continual learning strategies · synthetic continual learning | average loss | Helped |
| use a structured optimal protocol instead of random interleaving | multi-task learning from scratch · synthetic multi-task learning | average loss | Helped |
| use a structured optimal protocol instead of plain interleaving | multi-task learning from scratch · synthetic multi-task learning | average loss | Helped |
| use pseudo-optimal replay instead of no replay or interleaved replay | two-layer ReLU network · Fashion-MNIST continual learning | average loss | Helped |
| adapt between no replay and interleaved replay according to task performance and available samples | pseudo-optimal replay strategy · Fashion-MNIST continual learning | final average loss | Helped |
| use no replay instead of interleaved or pseudo-optimal replay | continual learning with small downstream tasks · Fashion-MNIST continual learning | final average loss | Helped |
| use no replay for longer downstream training | continual learning · Fashion-MNIST continual learning | average loss | Hurt |
Surprising findings
- Optimal replay reverses the non-monotonic dependence of forgetting on task similarity, with minimal error at intermediate similarity rather than maximal.
- On real Fashion-MNIST data, a simple pseudo-optimal focus-then-revision heuristic interpolates between no-replay and interleaved replay depending on downstream dataset size.
Limitations
- Pontryagin's maximum principle gives only a necessary condition for optimality, not a guaranteed global optimum.
- Analysis assumes i.i.d. Gaussian inputs and teacher-student setting with K=T=2 (or 3 in appendix), limiting generality.
- Task relative difficulty is not modelled.
Experimental setup
Datasets: Fashion-MNIST CIFAR10
Benchmarks: Fashion-MNIST CIFAR10
Metrics: generalisation error average test loss across tasks
Baselines: no replay interleaved replay randomly interleaved exponential learning rate schedule power-law learning rate schedule
Builds on: Pontryagin's maximum principle online SGD ODE framework for two-layer networks (Saad and Solla) teacher-student continual learning framework (Lee et al. 2021)
Closest papers by meaning
- Predicting the Susceptibility of Examples to Catastrophic Forgetting (ICML 2025 · similarity 0.89)
- When to Learn and When to Stop: Quitting at the Optimal Time (Student Abstract) (AAAI 2025 · similarity 0.89)
- Unlocking the Power of Rehearsal in Continual Learning: A Theoretical Perspective (ICML 2025 · similarity 0.89)
- STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.89)
- Mitigating Catastrophic Forgetting in Online Continual Learning by Modeling Previous Task Interrelations via Pareto Optimization (ICML 2024 · similarity 0.88)
- A Unified and General Framework for Continual Learning (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.