Research map / Label-efficient and robust learning / Domain adaptation and generalization
Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Decoupled Training (D-Train), a hyperparameter-free tri-phase general-to-specific training strategy on a shared-bottom model (pre-train shared, post-train multi-heads, fine-tune heads with frozen backbone) for multi-domain learning.
Problem
Multi-domain learning must handle dataset bias across domains and domain domination (imbalanced or long-tailed sample counts), while recent MDL methods add parameters and hyperparameters through complex architectures or losses.
Approach
Train a shared-bottom model in three phases: pre-train on all domains to warm up a root model, post-train with domain-specific heads, then freeze the backbone and fine-tune each domain's head independently so that head-dominant domains cannot dominate the others.
Key results
- On Office-Home (ResNet-50), D-Train reaches 86.0% average accuracy, above the best baseline PLE/MMoE at 85.2 and CDAN-MDL at 85.1.
- On DomainNet (ResNet-101), D-Train reaches 77.3% average and 72.2% worst-domain accuracy, versus 76.2 average for the best baseline CDAN-MDL.
- On FMoW (DenseNet-121), D-Train reaches 61.0% average accuracy and 58.3% worst-region accuracy, versus 60.4 and 57.2 for joint training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use D-Train instead of MMoE | Multi-domain learning with ResNet-50 · Office-Home | Avg. Acc.: 85.2 -> 86.0 | Helped |
| Use D-Train instead of CDAN-MDL | Multi-domain learning with ResNet-101 · DomainNet | Avg. Acc.: 76.2 -> 77.3 | Helped |
| Use D-Train instead of CDAN-MDL | Multi-domain learning with ResNet-101 · DomainNet | Worst Acc.: 71.0 -> 72.2 | Helped |
| Use D-Train instead of Jointly Train | Multi-domain learning with DenseNet-121 · FMoW | Avg. Acc.: 60.4 -> 61.0 | Helped |
| Use D-Train instead of MulANN | Multi-domain learning with DenseNet-121 · FMoW | Worst Acc.: 57.5 -> 58.3 | Helped |
| Add D-Train fine-tuning | PLE · Amazon Product Review | AUC_d: 77.62 -> 78.57 | Helped |
| Add D-Train fine-tuning | PLE · Amazon Product Review | AUC_s: 77.46 -> 77.56 | Helped |
| Add D-Train fine-tuning | MMoE · Amazon Product Review | AUC_d: 76.01 -> 77.70 | Helped |
| Add D-Train fine-tuning | MMoE · Amazon Product Review | AUC_s: 76.13 -> 77.05 | Helped |
| Add the fine-tuning phase (ablation) | D-Train · DomainNet | Avg. Acc.: 75.9 -> 77.3 | Helped |
| Add the post-training phase (ablation) | D-Train · DomainNet | Avg. Acc.: 76.6 -> 77.3 | Helped |
| Add the pre-training phase (ablation) | D-Train · DomainNet | Avg. Acc.: 77.0 -> 77.3 | Helped |
Surprising findings
- Separately training a model per domain outperforms jointly training on Office-Home, which the authors attribute to dataset bias obscuring domain distinctions.
- On DomainNet, several complex MDL methods (e.g., MoE at 74.6) underperform the simple shared-bottom baseline.
Limitations
- Gains on several Amazon domains are mixed (e.g., Electronics and TV drop for Shared Bottom + D-Train), and the online 'Music' gain is more modest than others.
- Improvements on some benchmarks are small (e.g., Office-Home, DomainNet worst-case gains of 1-2 points) and results are from single-setting runs with std reported only for DomainNet pre-train curves.
- Assumes the shared-bottom architecture with domain-specific heads and balanced per-domain batching; applicability to other architectures is not explored.
Experimental setup
Datasets: Office-Home DomainNet FMoW Amazon Product Review Tencent DSP online ads data
Benchmarks: Office-Home DomainNet FMoW Amazon Product Review
Metrics: accuracy worst-domain accuracy AUC online cost lift GMV lift
Baselines: Separately Train Jointly Train MulANN DANN-MDL ASP-MDL CDAN-MDL Shared Bottom MoE MMoE PLE
Builds on: shared-bottom architecture DANN CDAN MMoE PLE
Closest papers by meaning
- Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training (ICML 2025 · similarity 0.85)
- LFME: A Simple Framework for Learning from Multiple Experts in Domain Generalization (NeurIPS 2024 · similarity 0.84)
- Flatten Long-Range Loss Landscapes for Cross-Domain Few-Shot Learning (CVPR 2024 · similarity 0.84)
- PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization (CVPR 2024 · similarity 0.84)
- Mixup-Induced Domain Extrapolation for Domain Generalization (AAAI 2024 · similarity 0.84)
- Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.