Research map / Continual learning / Class-incremental
Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Finds that adding domain shift to class-incremental learning reduces forgetting, and proposes DisCo, a plug-and-play contrastive prototype-pool method that mimics this separation in feature space.
Problem
Catastrophic forgetting in class-incremental learning; the paper asks whether domain shift across tasks, which is usually considered harmful, actually reduces forgetting and how this effect can be exploited without task IDs at inference.
Approach
Observes that introducing per-task domain shift (CILD setting) lowers forgetting across many CIL methods, then imitates the effect in feature space: task-level and class-level triplet contrastive losses against a prototype pool (image or CLIP text prototypes) plus a cross-task contrastive distillation loss, added to existing CIL baselines.
Key results
- DisCo on iCaRL improves AA by 5.87% and reduces FM by 17.38% on CIFAR-100 (per paper text).
- On LwF, DisCo reduces FM by 22.09% on CIFAR-100 and 8.35% on Tiny-ImageNet.
- Under the CILD setting, DER's FM on DomainCIFAR-100 drops from 45.15 to 2.25 with ResNet-18 backbone (CIL vs CILD).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Introduce domain shift across tasks (CILD) instead of standard CIL | iCaRL · CIFAR-100 | FM: 59.42 -> 25.98(-33.44) | Helped |
| Introduce domain shift across tasks (CILD) instead of standard CIL | BiC · DomainNet | FM: 36.20 -> 13.12(-23.08) | Helped |
| Introduce domain shift across tasks (CILD) instead of standard CIL | L2P · CIFAR-100 | FM: 4.42 -> 13.46(+9.04) | Hurt |
| Add DisCo-I | iCaRL · CIFAR100 | AA: 64.24 -> 70.11 | Helped |
| Add DisCo-I | iCaRL · CIFAR100 | FM: 51.34 -> 33.96 | Helped |
| Use text prototypes rather than image prototypes | L2P + DisCo · CIFAR-100 | AA: 82.78 -> 83.12 | Helped |
| Add class-level regularization (Ccon) (ablation) | iCaRL + DisCo-I · CIFAR-100 | AA: 67.21 -> 70.11 | Helped |
| Add class-level regularization (Ccon) (ablation) | iCaRL + DisCo-I · CIFAR-100 | FM: 32.15 -> 33.96 | Hurt |
| Add task-level regularization (Tcon) (ablation) | iCaRL + DisCo-I · CIFAR-100 | AA: 66.00 -> 70.11 | Helped |
| Add task-level regularization (Tcon) (ablation) | iCaRL + DisCo-I · CIFAR-100 | FM: 47.41 -> 33.96 | Helped |
| Add cross-task contrastive distillation (CCD) (ablation) | iCaRL + DisCo-I · CIFAR-100 | AA: 69.35 -> 70.11 | Helped |
| Add cross-task contrastive distillation (CCD) (ablation) | iCaRL + DisCo-I · CIFAR-100 | FM: 35.47 -> 33.96 | Helped |
Surprising findings
- Adding domain shift across tasks substantially lowers forgetting for most CIL methods, contrary to the usual expectation that domain shift hurts.
- L2P is an exception: its forgetting increases under the domain-shift setting.
Limitations
- Performance gains are smaller on CUB200, where task boundaries are blurry.
- Task-level regularization lowers initial accuracy of new tasks, so AA gains are smaller than FM reductions.
- Prompt-based methods require adapting the regularization to prompt keys rather than CLS features.
Experimental setup
Datasets: CIFAR-100 Fashion-MNIST Tiny-ImageNet DomainNet CUB200
Benchmarks: CIFAR-100 Fashion-MNIST Tiny-ImageNet DomainNet CUB200
Metrics: average accuracy (AA) forgetting measure (FM) initial accuracy (IA) parameter interference value (PIV) parameter forward transfer score (PFTS)
Baselines: iCaRL BiC MEMO LwF DER L2P Co2L
Builds on: iCaRL BiC LwF DER L2P Co2L
Closest papers by meaning
- Defying Imbalanced Forgetting in Class Incremental Learning (AAAI 2024 · similarity 0.91)
- Continual Learning Using a Kernel-Based Method Over Foundation Models (AAAI 2025 · similarity 0.89)
- AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projection (NeurIPS 2025 · similarity 0.89)
- Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.89)
- Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings (NeurIPS 2024 · similarity 0.88)
- Gradual Divergence for Seamless Adaptation: A Novel Domain Incremental Learning Method (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.