Research map / Label-efficient and robust learning / Domain adaptation and generalization
A General Representation-Based Approach to Multi-Source Domain Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GAMA, a representation-based multi-source domain adaptation method that partitions the label's Markov blanket latents into parents, children, and spouses, with identifiability guarantees, implemented via hierarchical VAEs.
Problem
Multi-source unsupervised domain adaptation with high-dimensional observations, where existing latent-representation methods need restrictive assumptions (independent latents or invariant label distributions) for identifiability of the target joint distribution.
Approach
Show that learning the full label Markov blanket representation is underspecified for general shifts; instead partition the Markov blanket latents into label parents, children, and spouses, prove identifiability of these subspaces and of the target joint distribution under sufficient-change and low-dimensional-change assumptions, and implement with hierarchical VAEs.
Key results
- GAMA reaches 77.3% average accuracy on Office-Home, about 1 point above the strongest baseline iMSDA (76.2%).
- GAMA reaches 93.7% average accuracy on PACS, versus 93.5% for iMSDA.
- Ablation on Office-Home: removing all VAE losses (GAMA-vae) drops the average to 75.9%, and removing theta-related VAEs (GAMA-theta) drops it to 76.0%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GAMA instead of iMSDA | Office-Home · Office-Home Avg | accuracy: 76.2 -> 77.3 | Helped |
| use GAMA instead of iMSDA | PACS · PACS Avg | accuracy: 93.5 -> 93.7 | Helped |
| add VAE-related losses (ablation) | GAMA · Office-Home Avg | accuracy: 75.9 -> 77.3 | Helped |
| add theta-related VAE losses, L_ch, and L_Y (ablation) | GAMA · Office-Home Avg | accuracy: 76.0 -> 77.3 | Helped |
Surprising findings
- Learning representations of the full label Markov blanket is often underspecified for domain adaptation, motivating the split into parents, children, and spouses.
Limitations
- Evaluated only on Office-Home and PACS, which the authors themselves note as small-scale; larger datasets are future work.
- Gains over iMSDA are small (about 1 point on Office-Home, 0.2 on PACS average) and results are averaged over three seeds only.
- Requires sufficient changes across many source domains and intimate-neighbor structural assumptions.
Experimental setup
Datasets: Office-Home PACS
Benchmarks: Office-Home PACS
Metrics: accuracy
Baselines: Source Only DAN DANN MCD DANN+BSP M3SDA CMSS LtC-MSDA T-SVDNet WADN iMSDA
Builds on: iMSDA nonlinear ICA with sufficient changes Gumbel-Softmax VAE
Closest papers by meaning
- Domain Adaptation with Invariant Representation Learning: What Transformations to Learn? (NeurIPS 2021 · similarity 0.91)
- Visual Representation Learning over Latent Domains (ICLR 2022 · similarity 0.91)
- Learning Transferrable Representations for Unsupervised Domain Adaptation (NeurIPS 2016 · similarity 0.90)
- Mapping conditional distributions for domain adaptation under generalized target shift (ICLR 2022 · similarity 0.90)
- Partial disentanglement for domain adaptation (ICML 2022 · similarity 0.90)
- Adapting to Distribution Shift by Visual Domain Prompt Generation (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.