Research map / Continual learning / Online and replay-based
Layerwise Proximal Replay: A Proximal Point Method for Online Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Layerwise Proximal Replay (LPR), a layerwise preconditioned proximal-point optimizer applied on top of experience replay methods to stabilize optimization and improve online continual learning accuracy.
Problem
Replay-based online continual learning suffers from unstable optimization trajectories and underfitting on past data, with instabilities persisting even when the replay buffer stores all past data.
Approach
Derives a layerwise preconditioned gradient update from a proximal point formulation that constrains changes to replay-data hidden activations, with preconditioner P_l = I + w Z^T Z computed from replay buffer activations and refreshed periodically; it is a non-Euclidean proximal update combined with replay losses, distinct from gradient projection methods.
Key results
- On Split-CIFAR100 with memory size 1000, LPR (ER) improves final accuracy from 0.2262 to 0.2631 over ER, and LPR (DER) reaches 0.2634 vs 0.2339 for DER.
- With unlimited memory on Split-CIFAR100, LPR (ER) raises final accuracy from 0.3541 to 0.4244 (about 7 points).
- LPR improves all four replay methods (ER, DER, EMA, LODE) across memory sizes on the three benchmarks, with the largest gains on Split-CIFAR100; gains are larger on AAA and WC-Acc than on Acc.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LPR preconditioning to ER | ER · Split-CIFAR100, memory 2000 | Acc: 0.2937 { ± 0.0054} -> 0.3334 { ± 0.0056} | Helped |
| Add LPR preconditioning to ER | ER · Split-CIFAR100, memory 2000 | AAA: 0.3586 { ± 0.0048} -> 0.4251 { ± 0.0047} | Helped |
| Add LPR preconditioning to ER | ER · Split-CIFAR100, memory 2000 | WC-Acc: 0.1264 { ± 0.0030} -> 0.1926 { ± 0.0026} | Helped |
| Add LPR preconditioning to LODE | LODE · Split-TinyImageNet, memory 4000 | Acc: 0.2452 { ± 0.0022} -> 0.2518 { ± 0.0015} | Helped |
| Add LPR preconditioning to LODE | LODE · Split-TinyImageNet, memory 4000 | AAA: 0.3522 { ± 0.0041} -> 0.3592 { ± 0.0049} | Helped |
| Add LPR preconditioning to EMA | EMA · Online CLEAR, memory 2000 | Acc: 0.7357 { ± 0.0034} -> 0.7558 { ± 0.0013} | Helped |
| Add LPR preconditioning to EMA | EMA · Online CLEAR, memory 2000 | WC-Acc: 0.6145 { ± 0.0026} -> 0.6764 { ± 0.0038} | Helped |
| Add LPR preconditioning to ER with unlimited replay memory | ER · Split-CIFAR100, unlimited memory | Acc: 0.3541 { ± 0.0096} -> 0.4244 { ± 0.0051} | Helped |
| Add LPR preconditioning to ER with unlimited replay memory | ER · Split-TinyImageNet, unlimited memory | Acc: 0.3674 { ± 0.0038} -> 0.3686 { ± 0.0055} | No clear effect |
| Add LPR preconditioning to ER with unlimited replay memory | ER · Online CLEAR, unlimited memory | Acc: 0.7176 { ± 0.0062} -> 0.7411 { ± 0.0040} | Helped |
| Use T=5 and p=0.05 for preconditioner updating (ablation) | LPR (ER) · Split-CIFAR100, memory 2000 | accuracy: 0.2937 { ± 0.0054} -> 0.3307 { ± 0.0038} | Helped |
| Use T=5 and p=1.00 for preconditioner updating (ablation) | LPR (ER) · Split-TinyImageNet, memory 4000 | accuracy: 0.2192 { ± 0.0021} -> 0.2324 { ± 0.0019} | Helped |
Surprising findings
- Replay-based optimization instabilities persist even when the replay buffer stores all previous training examples, suggesting they are orthogonal to catastrophic forgetting.
- Proximal optimizers without replay perform very poorly, while replay plus SGD underperforms its LPR counterpart, indicating the two complement each other.
Limitations
- Requires periodic recomputation of layerwise preconditioners from replay buffer activations, adding wallclock overhead that grows with smaller update intervals and larger replay subsets.
- Gains are reported on only three image-classification benchmarks with a single ResNet18 architecture.
- Optimal omega_0 is dataset dependent and requires hyperparameter search on a single seed per dataset.
Experimental setup
Datasets: Split-CIFAR100 Split-TinyImageNet Online CLEAR
Benchmarks: Split-CIFAR100 Split-TinyImageNet Online CLEAR
Metrics: final accuracy (Acc) average anytime accuracy (AAA) worst-case accuracy (WC-Acc) representation drift total variation of accuracy
Baselines: ER DER EMA LODE A-GEM AOP
Builds on: experience replay dark experience replay exponential moving average replay loss decoupling (LODE) orthogonal gradient projection
Closest papers by meaning
- Online Continual Learning with Maximal Interfered Retrieval (NeurIPS 2019 · similarity 0.91)
- IDER: IDEMPOTENT EXPERIENCE REPLAY FOR RELIABLE CONTINUAL LEARNING (ICLR 2026 · similarity 0.91)
- Mitigating Catastrophic Forgetting in Online Continual Learning by Modeling Previous Task Interrelations via Pareto Optimization (ICML 2024 · similarity 0.90)
- Predicting the Susceptibility of Examples to Catastrophic Forgetting (ICML 2025 · similarity 0.90)
- Online Curvature-Aware Replay: Leveraging $\mathbf{2^{nd}}$ Order Information for Online Continual Learning (ICML 2025 · similarity 0.90)
- When to Learn and When to Stop: Quitting at the Optimal Time (Student Abstract) (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.