Research map / Continual learning / Domain and prompt-based
Boosting Domain Incremental Learning: Selecting the Optimal Parameters is All You Need
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SOYO, a trainable lightweight domain-label predictor for parameter-isolation domain incremental learning that uses a Gaussian Mixture Compressor, Domain Feature Resampler, and multi-level feature fusion to select PEFT parameters per domain across classification, detection, and speech enhancement.
Problem
Parameter-isolation domain incremental learning (PIDIL) methods degrade at inference when the domain label of a test sample must be predicted, because training-free domain selectors lose accuracy as the number of domains grows; prior-domain data is also unavailable for balanced training.
Approach
Train a small domain classifier (MDFN) on fused shallow and deep ViT features to predict the domain and select the matching PEFT parameters, balancing it with previous-domain features that are compressed into a Gaussian mixture (GMC) and resampled (DFR) instead of stored raw.
Key results
- On DomainNet with ViT-B, SOYO raised PINA-D average accuracy from 62.23% to 65.25% and parameter selection accuracy from 80.55% to 85.37%.
- On Pascal VOC series detection, LDB+SOYO reached 69.2/65.3/59.6 mAP across sessions, with session 4 selection accuracy rising from 77.1% (NMC) to 96.7%.
- On speech enhancement (WSJ0), LNA+SOYO improved average SI-SNR from 16.07 to 17.50 and PESQ from 3.13 to 3.29.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SOYO to PINA-D | PINA-D with CLIP · DomainNet | AT: 73.27 -> 74.72 | Helped |
| Add SOYO to PINA-D | PINA-D with CLIP · CDDB-Hard | AT: 89.03 -> 90.12 | Helped |
| Add SOYO to PINA-D | PINA-D with CLIP · CORe50 | AT: 89.23 -> 89.82 | Helped |
| Add SOYO to LDB | LDB · Pascal VOC series, Session 4 | mAP: 56.8 -> 59.6 | Helped |
| Add SOYO to LDB | LDB · BDD100K series, Session 3 | mAP: 51.1 -> 51.7 | Helped |
| Add SOYO to LNA | LNA · WSJ0 synthetic, Avg. | SI-SNR (dB): 16.07 -> 17.50 | Helped |
| Add SOYO to LNA | LNA · WSJ0 synthetic, Avg. | SDR (dB): 16.41 -> 17.87 | Helped |
| Add SOYO to LNA | LNA · WSJ0 synthetic, Avg. | PESQ: 3.13 -> 3.29 | Helped |
| Use GMC with K = 2 instead of K = 1 (ablation) | SOYO · DomainNet | AT: 64.91 -> 65.25 | Helped |
| Replace GMC with PCA using N = 10 (ablation) | SOYO · DomainNet | AT: 65.25 -> 63.74 | Hurt |
| Replace GMC with PCA using N = 100 (ablation) | SOYO · DomainNet | AT: 65.25 -> 64.30 | Hurt |
| Replace GMC with saved means and standard deviations (ablation) | SOYO · DomainNet | AT: 65.25 -> 63.48 | Hurt |
Surprising findings
- Reported that SOYO's domain-selection gains translate to accuracy near the oracle upper bound on Pascal VOC (0.3% gap), suggesting selection errors were the main bottleneck.
Limitations
- Training-based selector adds storage and training overhead relative to training-free selectors.
- Evaluated mainly on ViT-B and CLIP backbones; speech experiments use a synthetic dataset.
Experimental setup
Datasets: DomainNet CDDB-Hard CORe50 Pascal VOC 2007 Clipart Watercolor Comic BDD100K Cityscapes Rainy Cityscapes WSJ0 NOISEX-92
Benchmarks: DomainNet CDDB-Hard CORe50 Pascal VOC series BDD100K series WSJ0 synthetic speech dataset
Metrics: average accuracy forgetting parameter selection accuracy mAP SI-SNR SDR PESQ
Baselines: S-Prompts PINA PINA-D LDB LNA MoP-CLIP DyTox L2P DualPrompt CODA-P
Builds on: S-Prompts PINA LDB LNA
Closest papers by meaning
- Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models (NeurIPS 2024 · similarity 0.83)
- Self-Training with Dynamic Weighting for Robust Gradual Domain Adaptation (NeurIPS 2025 · similarity 0.83)
- Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learning (AAAI 2025 · similarity 0.82)
- SD-LoRA: Scalable Decoupled Low-Rank Adaptation for Class Incremental Learning (ICLR 2025 · similarity 0.82)
- Attraction Diminishing and Distributing for Few-Shot Class-Incremental Learning (CVPR 2025 · similarity 0.82)
- GraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.