Research map / Label-efficient and robust learning / Domain adaptation and generalization
Leveraging Normalization Layer in Adapters with Progressive Learning and Adaptive Distillation for Cross-Domain Few-Shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ProLAD, which pairs a normalization-free linear adapter with a normalization-layer adapter, trained progressively with a domain-similarity-weighted distillation to select the primary adapter for cross-domain few-shot learning.
Problem
Cross-domain few-shot learning must adapt a model trained on base classes to novel domains with few samples; adapters ignore batch-statistic shift from the source and noisy batch statistics from few-shot target data.
Approach
Use two adapters, one without normalization (better for similar domains) and one with a standard normalization layer plus group convolution (better for dissimilar domains), trained progressively in two stages, with a distillation coefficient adaptively set from a domain-similarity estimate computed from support-set feature/prototype cosine similarity differences (or support loss/accuracy) to switc
Key results
- In the SDL setting on Meta-Dataset, ProLAD-sim reaches 74.7% average accuracy versus 72.4% for TSA and 57.1% for the best non-adapter baseline context in the table.
- The normalization-layer adapter (ProLAD's SN-based branch) gains +4.1% on dissimilar domains over the normalization-free adapter, while the normalization-free adapter gains +2.8% on similar domains.
- In the MDL setting, ProLAD-sim achieves 80.0% average accuracy versus 79.0% for TSA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ProLAD-sim instead of TSA | ResNet18 few-shot adaptation · Meta-Dataset SDL, Avg. all | accuracy: 72.4 -> 74.7 | Helped |
| use ProLAD-sim instead of TSA | ResNet18 few-shot adaptation · Meta-Dataset MDL, Avg. all | accuracy: 79.0 -> 80.0 | Helped |
| add the normalization adapter (ablation) | the no-normalization adapter · Meta-Dataset SDL, Avg. dissimilar | accuracy: 81.7 -> 85.8 | Helped |
| use the no-normalization adapter rather than the normalization adapter (ablation) | single-adapter adaptation · Meta-Dataset SDL, Avg. similar | accuracy: 65.2 -> 68.0 | Helped |
| add adaptive distillation to progressive learning (ablation) | training both adapters · Meta-Dataset SDL, Avg. all | accuracy: 72.4 -> 74.7 | Helped |
| add progressive learning to distillation and adaptive-coefficient training (ablation) | training both adapters · Meta-Dataset SDL, Avg. all | accuracy: 73.9 -> 74.7 | Helped |
| add standard normalization to group convolution (ablation) | the normalization adapter · Meta-Dataset SDL, Avg. all | accuracy: 72.6 -> 74.7 | Helped |
| replace convolution with group convolution after standard normalization (ablation) | the normalization adapter · Meta-Dataset SDL, Avg. all | accuracy: 74.1 -> 74.7 | Helped |
| replace FiLM with group convolution after standard normalization (ablation) | the normalization adapter · Meta-Dataset SDL, Avg. all | accuracy: 73.1 -> 74.7 | Helped |
| add standard normalization to the FiLM adapter (ablation) | the normalization adapter · Meta-Dataset SDL, Avg. all | accuracy: 73.0 -> 73.1 | No clear effect |
| train both adapters from scratch rather than progressively (ablation) | two-adapter training · Meta-Dataset SDL, Avg. all | accuracy: 74.7 -> 72.0 | Hurt |
| use a support-loss-and-accuracy coefficient instead of a feature-similarity coefficient (ablation) | ProLAD · Meta-Dataset SDL, Avg. all | accuracy: 74.7 -> 74.0 | Hurt |
Surprising findings
- Training both adapters from scratch without distillation yields performance on par with the normalization-layer adapter alone, suggesting the normalization adapter dominates activation.
Limitations
- Adapter selection depends on a heuristic domain-similarity estimate and a scaling factor beta tuned via hyperparameter search.
- Fixed coefficients and the loss/accuracy variant underperform the similarity-difference variant on some domains.
- Assumes no unlabeled test-domain data is available during pretraining.
Experimental setup
Datasets: ImageNet Meta-Dataset
Benchmarks: Meta-Dataset
Metrics: mean accuracy 95% confidence interval
Baselines: Finetune Protonet Proto-MAML BOHB Simple CNAPS FLUTE TSA URL URT SUR tri-M Transductive CNAPS
Builds on: TSA URL Simple CNAPS Protonet
Closest papers by meaning
- Cross-Domain Few-Shot Segmentation via Multi-view Progressive Adaptation (CVPR 2026 · similarity 0.89)
- Discriminative Sample-Guided and Parameter-Efficient Feature Space Adaptation for Cross-Domain Few-Shot Learning (CVPR 2024 · similarity 0.88)
- Reducing Divergence in Batch Normalization for Domain Adaptation (AAAI 2025 · similarity 0.88)
- Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization (AAAI 2024 · similarity 0.87)
- Few-shot Learner Parameterization by Diffusion Time-steps (CVPR 2024 · similarity 0.87)
- Adapter Naturally Serves as Decoupler for Cross-Domain Few-Shot Semantic Segmentation (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.