Research map / Label-efficient and robust learning / Domain adaptation and generalization
Agile Multi-Source-Free Domain Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Bi-ATEN, a bi-level attention ensemble that adaptively combines multiple frozen source models for source-free multi-source domain adaptation by tuning only source bottlenecks, reaching SOTA on DomainNet with under 3% trainable parameters.
Problem
Multi-source-free domain adaptation requires tuning every source model, which is computationally expensive as the number of source domains or model size grows; existing methods also learn only domain-level ensemble weights that ignore instance specificity.
Approach
Keep source backbones and classifiers frozen, and learn intra-domain weights (matching each bottleneck feature to the most compatible cross-domain classifier via cosine attention) plus inter-domain instance-specific ensemble weights via attention, trained with information-maximization and dynamic-cluster pseudo labels; a simplified ATEN variant plugs into existing MSFDA methods.
Key results
- Bi-ATEN reaches 59.6% average accuracy on DomainNet (with 6 target domains, SwinTransformer backbone) versus 59.1% for PMTrans with 447.4M params, using 10.56M trainable parameters.
- Plugging ATEN into existing MSFDA methods yields up to 4.1% average accuracy gain on Office-Home and Office-Caltech; CAiDA+ATEN reaches 98.0% on Office-Caltech.
- ATEN achieves 970 throughput versus 46 for PMTrans on DomainNet, per the paper's comparison table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Bi-ATEN instead of ATEN by adding intra-domain weights | ATEN on DomainNet · DomainNet Avg. | accuracy: 59.1 -> 59.6 | Helped |
| Equip DATE with ATEN | DATE · Office-Home Avg. | accuracy: 76.3 -> 77.1 | Helped |
| Equip DECISION with ATEN | DECISION · Office-Home Avg. | accuracy: 74.7 -> 76.3 | Helped |
| Equip CAiDA with ATEN | CAiDA · Office-Home Avg. | accuracy: 72.2 -> 76.3 | Helped |
| Equip CAiDA with ATEN | CAiDA · Office-Caltech Avg. | accuracy: 97.1 -> 98.0 | Helped |
| Replace PMTrans with Bi-ATEN | DomainNet target clp · DomainNet target clp | accuracy: 74.1 -> 77.0 | Helped |
| Add intra-domain weights to ATEN (ablation) | Bi-ATEN on DomainNet · DomainNet Avg. (three-task ablation table) | accuracy: 59.5 -> 60.1 | Helped |
| Add alternate training (ablation) | Bi-ATEN on DomainNet · DomainNet Avg. (three-task ablation table) | accuracy: 59.5 -> 60.1 | Helped |
| Remove alternate training (ablation) | Bi-ATEN on DomainNet target inf · DomainNet target inf | accuracy: 38.5 -> 38.6 | Helped |
| Add intra-domain loss (ablation) | Bi-ATEN on DomainNet · DomainNet Avg. (three-task ablation table) | accuracy: 58.4 -> 60.1 | Helped |
| Add IM loss (ablation) | Bi-ATEN on DomainNet · DomainNet Avg. (three-task ablation table) | accuracy: 59.3 -> 60.1 | Helped |
Surprising findings
- Bi-ATEN underperforms PMTrans on the pnt and qdr targets, which the authors attribute to PMTrans using labeled source data.
Limitations
- Bi-ATEN trains more parameters and is slower than ATEN (1.2H vs 0.6H training time), and it trails PMTrans on some DomainNet targets.
- Results on the pnt and qdr targets are behind PMTrans, which uses labeled source data.
- Hyperparameters (lambda, gamma, epochs, learning rate) are tuned per target domain, and the optimal combination varies across targets.
Experimental setup
Datasets: DomainNet Office-Home Office-Caltech
Benchmarks: DomainNet Office-Home Office-Caltech
Metrics: accuracy throughput trainable parameters training time
Baselines: DECISION CAiDA DATE PMTrans DRT STEM M3SDA LtC-MSDA TransMDA Surrogate AVG-ENS CDTrans
Builds on: SHOT DECISION CAiDA DATE PMTrans
Closest papers by meaning
- Slimmable Domain Adaptation (CVPR 2022 · similarity 0.88)
- Cycle Self-Refinement for Multi-Source Domain Adaptation (AAAI 2024 · similarity 0.87)
- Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer (CVPR 2024 · similarity 0.87)
- Revisiting Source-Free Domain Adaptation: Insights into Representativeness, Generalization, and Variety (CVPR 2025 · similarity 0.86)
- Unveiling the Unknown: Unleashing the Power of Unknown to Known in Open-Set Source-Free Domain Adaptation (CVPR 2024 · similarity 0.86)
- Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.