Research map / Continual learning / Online and replay-based
Continual Normalization: Rethinking Batch Normalization for Online Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace BN with CN | ER on Split CIFAR100 · Split CIFAR100 | ACC: 64.97$\pm$1.09 -> 67.48$\pm$0.81 | Helped |
| Replace BN with CN | ER on Split Mini IMN · Split Mini IMN | ACC: 59.09$\pm$1.74 -> 64.28$\pm$1.49 | Helped |
| Replace BN with CN | DER++ on Split CIFAR100 · Split CIFAR100 | ACC: 66.50$\pm$2.52 -> 69.13$\pm$0.56 | Helped |
| Replace BN with CN | DER++ on Split Mini IMN · Split Mini IMN | ACC: 61.08$\pm$0.91 -> 66.29$\pm$1.11 | Helped |
| Use global moments from all tasks at test time instead of running BN moments | Single-BN on pMNIST · pMNIST | FM: 18.65 -> 15.13 | Helped |
| Use global moments from all tasks at test time instead of running BN moments | ER-BN on pMNIST · pMNIST | ACC: 80.66 -> 81.75 | Helped |
| Replace BN with CN | DER++ with buffer 5120 · Split CIFAR-10, Class-IL | ACC: 52.0$\pm$7.8 -> 57.9$\pm$4.1 | Helped |
| Replace BN with CN | PRS · NUS-WIDEseq, minority classes | C-F1: 28.5 -> 27.3 | Hurt |
| Reverse the normalization order from GN→BN to BN→GN (ablation) | CN · COCOseq, overall | O-F1: 49.8 -> 47.9 | Hurt |
| Add affine weights to the first GN normalization step (ablation) | CN · COCOseq, overall | O-F1: 49.8 -> 47.7 | Hurt |
| Replace EMA with cumulative moving average (ablation) | ER+BN · Split CIFAR-100 | ACC: 64.97$\pm$1.09 -> 35.86$\pm$0.90 | Hurt |
| Reduce the BN EMA update rate to η=0.01 (ablation) | ER+BN · Split CIFAR-100 | FM: 9.24$\pm$1.98 -> 6.32$\pm$2.32 | Helped |
Closest papers by meaning
- A Statistical Theory of Regularization-Based Continual Learning (ICML 2024 · similarity 0.89)
- The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting — An Analytical Model (ICLR 2024 · similarity 0.89)
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning (NeurIPS 2025 · similarity 0.88)
- Optimal Task Order for Continual Learning of Multiple Tasks (ICML 2025 · similarity 0.88)
- A Unified and General Framework for Continual Learning (ICLR 2024 · similarity 0.88)
- Optimal Rates in Continual Linear Regression via Increasing Regularization (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.