Research map / Label-efficient and robust learning / Domain adaptation and generalization
Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts
Tao Zhong, Zhixiang Chi, Li Gu, Yang Wang, YUANHAO YU, Jin Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| mask the in-domain expert during meta-training (ablation) | Meta-DMoE · iWildCam OOD | accuracy: 74.1 -> 77.2 | Improved |
| mask the in-domain expert during meta-training (ablation) | Meta-DMoE · iWildCam ID test | accuracy: 76.4 -> 72.9 | Hurt |
| increase number of experts from 2 to 10 (ablation) | Meta-DMoE · iWildCam | accuracy: 70.4 -> 77.2 | Improved |
| meta-learn the aggregator instead of pretraining it (ablation) | Meta-DMoE · iWildCam | accuracy: 74.8 -> 77.2 | Improved |
| meta-train the student instead of random init (ablation) | Meta-DMoE · iWildCam | accuracy: 32.7 -> 77.2 | Improved |
| use transformer aggregator instead of average pooling (ablation) | Meta-DMoE · iWildCam | accuracy: 69.7 -> 77.2 | Improved |
| use transformer aggregator instead of MLP projector (ablation) | Meta-DMoE · iWildCam | accuracy: 73.7 -> 77.2 | Improved |
| distill features only instead of logits (ablation) | Meta-DMoE · iWildCam | accuracy: 72.1 -> 77.2 | Improved |
| distill features only instead of logits plus features (ablation) | Meta-DMoE · iWildCam | accuracy: 73.1 -> 77.2 | Improved |
| increase adaptation images from 2 to 24 (ablation) | Meta-DMoE · iWildCam | accuracy: 76.5 -> 77.2 | Improved |
| ResNet-50 experts instead of MobileNet V2 experts (ablation) | Meta-DMoE · iWildCam (MobileNet V2 student) | accuracy / Macro F1: 59.5 -> 58.8 acc; 19.7 -> 21.0 F1 | Mixed |
| replace ARM-BN with Meta-DMoE | ARM-BN · RxRx1 | accuracy: 31.2 -> 29.8 | Hurt |
Later papers that compared with this method
2 later accepted papers report Meta-DMoE as a baseline in their results tables; 2 of them report a better number on most of the results they share with it.
- Unified Modality Separation: A Vision-Language Framework for Unsupervised Domain Adaptation (TPAMI 2025 · better on 1 of 1 shared result)
- Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization (AAAI 2024 · better on 6 of 6 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Adapting to Distribution Shift by Visual Domain Prompt Generation (ICLR 2024 · similarity 0.88)
- Reinforced Cross-Domain Knowledge Distillation on Time Series Data (NeurIPS 2024 · similarity 0.88)
- Class-aware Domain Knowledge Fusion and Fission for Continual Test-Time Adaptation (NeurIPS 2025 · similarity 0.87)
- BECoTTA: Input-dependent Online Blending of Experts for Continual Test-time Adaptation (ICML 2024 · similarity 0.87)
- Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization (AAAI 2024 · similarity 0.86)
- Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
