Research map / Label-efficient and robust learning / Domain adaptation and generalization
Understanding and Improving Source-free Domain Adaptation from a Theoretical Perspective
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that discriminability and diversity losses in SFDA mirror the self-training objective under an expansion assumption with a target error bound, and uses these insights to build an improved SFDA method with auto-adjusting diversity weight and learnable teacher-student augmentation.
Problem
Source-free domain adaptation (SFDA) methods work well empirically, but lack a general theoretical explanation of why they succeed.
Approach
Maps the discriminability and diversity losses common to SFDA methods onto the self-training objective (consistency term plus diversity constraint) from the expansion-assumption theory, yielding a target error bound; the insights motivate auto-adjusting the diversity weight by the discriminability loss and training a learnable augmentation with a teacher-student framework that tightens the bound.
Key results
- Improved SFDA reaches 90.3% average accuracy on Office-31 (+0.4 over AaD), 73.4% on Office-Home (+0.7 over AaD), and 88.4% per-class on VisDA2017 (+0.4 over AaD).
- Applying the proposed techniques to SHOT-IM improves it by +1.3 on Office-31, +0.7 on Office-Home, and +2.9 on VisDA2017.
- On the synthetic 2D experiment, adding the diversity loss raises accuracy from 93.0% (discriminability only) to 95.3%, and decaying the diversity weight raises it to 95.7%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a diversity loss to discriminability-only training (ablation) | SFDA training on synthetic inter-twinning moons · Synthetic inter-twinning moons | accuracy: 93.0 -> 95.3 | Helped |
| decay the diversity-loss trade-off during training (ablation) | SFDA training on synthetic inter-twinning moons · Synthetic inter-twinning moons | accuracy: 95.3 -> 95.7 | Helped |
| train learnable augmentation with the teacher-student framework (ablation) | AaD-based Improved SFDA with random augmentation initialization · Office-Home | accuracy gain over AaD: + 0.4 -> + 0.6 | Helped |
| train learnable augmentation with the teacher-student framework (ablation) | AaD-based Improved SFDA with AutoAugment initialization · Office-Home | accuracy gain over AaD: + 0.6 -> + 0.7 | Helped |
| automatically adjust rather than fix the diversity constraint (ablation) | AaD-based Improved SFDA · Office-Home | accuracy: 69.3, 72.8, 72.6, 71.1 -> 73.4 | Helped |
| apply the proposed techniques to SHOT-IM | SHOT-IM · Office-31 | accuracy: 87.3 -> 88.6 (+1.3) | Helped |
| apply the proposed techniques to SHOT-IM | SHOT-IM · Office-Home | accuracy: 70.5 -> 71.2 (+0.7) | Helped |
| apply the proposed techniques to SHOT-IM | SHOT-IM · VisDA2017 | per-class accuracy: 80.4 -> 83.3 (+2.9) | Helped |
| use Improved SFDA instead of AaD | SFDA on Office-31 · Office-31 | accuracy: 89.9 -> 90.3 (+0.4) | Mixed |
| use Improved SFDA instead of AaD | SFDA on Office-Home · Office-Home | accuracy: 72.7 -> 73.4 (+0.7) | Helped |
| use Improved SFDA instead of AaD | SFDA on VisDA2017 · VisDA2017 | per-class accuracy: 88.0 -> 88.4 (+0.4) | Helped |
Surprising findings
- Fixed diversity weights performed notably worse than the auto-adjusting one; λmax_div=0.1 gave 69.3% vs 73.4% for the auto-adjusting variant on Office-Home.
- Accuracy deteriorates more sharply when the diversity coefficient is too small than when it is too large.
Limitations
- Does not account for how well the source model performs on the target domain before adaptation.
- Hyperparameters (λmax_div, λF, λA, ê) are tuned per dataset, and gains over the AaD baseline are modest (0.4-0.7 points).
Experimental setup
Datasets: Office-31 Office-Home VisDA2017
Benchmarks: Office-31 Office-Home VisDA2017
Metrics: classification accuracy per-class accuracy
Baselines: SHOT-IM AaD 3C-GAN VDM-DA A2Net NRC CPGA CoWA-JMDS C-SFDA DaC AdaContrast G-SFDA
Builds on: AaD SHOT-IM Faster AutoAugment theory of self-training with expansion assumption
Closest papers by meaning
- Revisiting Source-Free Domain Adaptation: Insights into Representativeness, Generalization, and Variety (CVPR 2025 · similarity 0.92)
- Proxy Denoising for Source-Free Domain Adaptation (ICLR 2025 · similarity 0.90)
- Dynamic Target Distribution Estimation for Source-Free Open-Set Domain Adaptation (AAAI 2025 · similarity 0.89)
- On $f$-Divergence Principled Domain Adaptation: An Improved Framework (NeurIPS 2024 · similarity 0.89)
- SF(DA)$^2$: Source-free Domain Adaptation Through the Lens of Data Augmentation (ICLR 2024 · similarity 0.89)
- Diffusion-Driven Progressive Target Manipulation for Source-Free Domain Adaptation (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.