Research map / Label-efficient and robust learning / Domain adaptation and generalization
On $f$-Divergence Principled Domain Adaptation: An Improved Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Refines the f-divergence domain adaptation framework of Acuna et al. (f-DAL) using a tighter Legendre-type variational form, introduces f-DD (no absolute value, with scaling parameter), and derives target error, sample-complexity, and localized fast-rate bounds, with improved UDA benchmark results.
Problem
Existing f-divergence-based domain adaptation theory (f-DAL) relies on a weak variational representation and an absolute value function that creates a gap between theory and algorithm, and cannot recover the DV representation of KL; its bounds also converge at slow rates.
Approach
Uses the shift-based variational representation of f-divergence to define f-DD, a hypothesis-specific discrepancy without absolute value but with a scaling parameter t, yielding target error bounds, KL-recovering results, and localized fast-rate bounds; the algorithm optimizes f-DD directly to match the theory.
Key results
- Jeffreys-DD reaches 90.1% average accuracy on Office-31 vs 89.5% for f-DAL.
- Jeffreys-DD reaches 70.2% average on Office-Home vs 68.5% for f-DAL.
- On Digits, Jeffreys-DD averages 97.1% vs 96.3% for f-DAL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Jeffreys-DD instead of f-DAL | UDA domain-adversarial learning · Office-31 | accuracy (%): 89.5 -> 90.1 | Helped |
| use Jeffreys-DD instead of f-DAL | UDA domain-adversarial learning · Office-Home | accuracy (%): 68.5 -> 70.2 | Helped |
| use Jeffreys-DD instead of f-DAL | UDA domain-adversarial learning · Digits | accuracy (%): 96.3 -> 97.1 | Helped |
| use KL-DD instead of χ²-DD | f-DD domain-adversarial learning · Office-31 | accuracy (%): 89.7 -> 89.8 | Helped |
| use KL-DD instead of χ²-DD | f-DD domain-adversarial learning · Office-Home | accuracy (%): 69.2 -> 69.4 | Helped |
| use KL-DD instead of χ²-DD | f-DD domain-adversarial learning · Digits | accuracy (%): 96.4 -> 96.9 | Helped |
| add the absolute value function to the discrepancy (ablation) | χ²-based domain-adversarial learning · Office-31 A→D and Office-Home Ar→Cl | estimated f-divergence | Hurt |
| optimize t using the quadratic approximation instead of fixing t=1 (ablation) | KL-DD · Office-31 | accuracy (%): 89.8 -> 89.6 | Hurt |
| optimize t using the quadratic approximation instead of fixing t=1 (ablation) | KL-DD · Office-Home | accuracy (%): 69.4 -> 69.2 | Hurt |
| optimize t using the quadratic approximation instead of fixing t=1 (ablation) | KL-DD · Digits | accuracy (%): 96.9 -> 96.5 | Hurt |
Surprising findings
- Absolute-value discrepancy can explode during training, overestimating f-divergence.
- Under the tighter representation, chi^2-DD is no longer superior to KL-DD; KL-DD slightly outperforms chi^2-DD on all three benchmarks.
- Optimizing the scaling parameter t gives no improvement over fixing t=1 on these benchmarks.
Limitations
- Fast-rate bound relies on conditions on constants C1, C2 and a localized Rashomon set that may be hard to verify.
- Lipschitz/boundedness assumptions (loss in [0,1]) needed for several bounds.
- Experimental gains are modest (e.g., 0.6 points on Office-31 average).
Experimental setup
Datasets: Office-31 Office-Home MNIST USPS
Benchmarks: Office-31 Office-Home Digits (MNIST-USPS)
Metrics: accuracy
Baselines: ResNet-50 DANN MDD KL f-DAL
Builds on: f-DAL DANN Margin Disparity Discrepancy Donsker-Varadhan representation
Closest papers by meaning
- Domain Adaptation with Adaptive $f$-Divergence: Tighter Variational Representation and Generalization Bounds (ICML 2026 · similarity 0.94)
- Bridging Theory and Algorithm for Domain Adaptation (ICML 2019 · similarity 0.90)
- f-Domain Adversarial Learning: Theory and Algorithms (ICML 2021 · similarity 0.90)
- Understanding and Improving Source-free Domain Adaptation from a Theoretical Perspective (CVPR 2024 · similarity 0.89)
- Denoised Maximum Classifier Discrepancy for Source-Free Unsupervised Domain Adaptation (AAAI 2022 · similarity 0.89)
- Can We Evaluate Domain Adaptation Models Without Target-Domain Labels? (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.