Research map / Continual learning / Domain and prompt-based
Gradual Divergence for Seamless Adaptation: A Novel Domain Incremental Learning Method
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DARE, a three-stage (Divergence, Adaptation, Refinement) domain-incremental learning method with dual classifiers and an Intermediary Reservoir Sampling buffer strategy that reduces representation drift at task boundaries.
Problem
Domain incremental learning suffers from catastrophic forgetting, where representation drift of previously learned clusters at task boundaries causes accuracy loss on old domains; prior rehearsal methods do not explicitly address this drift.
Approach
Gradually adapt new-domain representations into the feature space spanned by old-domain samples using two classifiers (cross-entropy and supervised contrastive) whose discrepancy is maximized (Divergence) and then minimized through the encoder (Adaptation), followed by Refinement; plus an Intermediary Reservoir Sampling buffer that samples logits from a normal distribution centered on mid-task tra
Key results
- On DN4IL with buffer size 50, DARE reaches 32.32% last accuracy versus 28.08% for DER++ and 24.24% for ER.
- On iCIFAR-20 with buffer size 200, DARE achieves 58.53% last accuracy versus 58.39% for DER++, with BWT of -30.22 vs -33.61.
- DARE++ (multi-memory variant) with buffer 50 reaches 40.51% on DN4IL, comparable to CLS-ER and DUCA with fewer parameters (22.5M vs 33.8M).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DARE rather than DER++ | ResNet-18 continual learner · DN4IL, buffer 50 | Last Accuracy: 28.08 -> 32.32 | Helped |
| Use DARE rather than DER++ | ResNet-18 continual learner · iCIFAR-20, buffer 100 | BWT: -37.33 -> -33.20 | Helped |
| Use Intermediary Reservoir Sampling instead of Reservoir Sampling | DARE · DN4IL, buffer 50 | Last Accuracy: 36.17 -> 40.59 | Helped |
| Use Intermediary Reservoir Sampling instead of Reservoir Sampling | DER++ · DN4IL, buffer 50 | Last Accuracy: 35.74 -> 37.60 | Helped |
| Add L1 (ablation) | DARE · DN4IL, buffer 50 | Last Accuracy: 11.22 -> 32.32 | Helped |
| Add L2 (ablation) | DARE · DN4IL, buffer 50 | Last Accuracy: 31.11 -> 32.32 | Helped |
| Add L3 (ablation) | DARE · DN4IL, buffer 50 | Last Accuracy: 24.69 -> 32.32 | Helped |
| Add L4 (ablation) | DARE · DN4IL, buffer 50 | Last Accuracy: 31.63 -> 32.32 | Helped |
| Add L1 (ablation) | DARE · DN4IL, buffer 200 | Last Accuracy: 22.26 -> 40.59 | Helped |
| Add L2 (ablation) | DARE · DN4IL, buffer 200 | Last Accuracy: 38.69 -> 40.59 | Helped |
| Add L3 (ablation) | DARE · DN4IL, buffer 200 | Last Accuracy: 24.55 -> 40.59 | Helped |
| Add L4 (ablation) | DARE · DN4IL, buffer 200 | Last Accuracy: 38.92 -> 40.59 | Helped |
Surprising findings
- Joint training gains are not reached; the gap between DARE and joint on DN4IL remains large (32.32 vs 59.93 at buffer 50).
- Regularization methods oEWC and SI perform poorly in DIL, and A-GEM and FDR perform comparably to ER.
Limitations
- Relies on task-id to trigger the IRS buffer sampling strategy, limiting task-agnostic use.
- Gap to joint training upper bound remains large on DN4IL.
- Evaluated only on two DIL benchmarks at 64x64 and 32x32 resolution with ResNet-18.
Experimental setup
Datasets: DN4IL iCIFAR-20 CIFAR-100 DomainNet
Benchmarks: DN4IL iCIFAR-20
Metrics: last accuracy backward transfer (BWT) number of parameters calibration error logit norm
Baselines: ER DER++ CLS-ER DUCA oEWC SI A-GEM FDR SGD Joint
Builds on: Dark Experience Replay (DER++) CLS-ER DUCA Supervised contrastive learning Mammoth
Closest papers by meaning
- Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learning (AAAI 2025 · similarity 0.88)
- Overcoming Catastrophic Forgetting in Visual Continual Learning with Reinforcement Fine-Tuning (NeurIPS 2026 · similarity 0.87)
- Dual Consolidation for Pre-Trained Model-Based Domain-Incremental Learning (CVPR 2025 · similarity 0.87)
- Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.87)
- PaLoRA: Paced Low-Rank Adaptation for Continual Learning (NeurIPS 2026 · similarity 0.87)
- Fine-Grained Knowledge Selection and Restoration for Non-exemplar Class Incremental Learning (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.