Research map / Label-efficient and robust learning / Domain adaptation and generalization
MoE^2: A Mixture-of-Mixtures of Experts for Ensemble-Free Domain Generalization
Ahmed Radwan, Mahmoud Soliman, Omar Abdelaziz, Ahmad Abdel-Qader, Mohamed S. Shehata
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| synthesize instance-specific adapters from a bank of MoA experts (MoE2) instead of a single static adapter | frozen CLIP ViT-B/16 · DG benchmarks average | average accuracy (%): 74.1 -> 78.1 (ERM Attention to MoE2) | Improved |
| replace full fine-tuning with MoE2 adapters on frozen backbone | CLIP ViT-B/16 · DG benchmarks average | average accuracy (%): 67.1 -> 78.1 | Improved |
| use single MoE2 model instead of large model-pool ensemble | SIMPLE+ · DG benchmarks average | average accuracy (%): 78.1 -> 78.1 | No clear effect |
| increase MoA experts per layer in a standard mixture of adapters (ablation) | MoA · OfficeHome | average accuracy (%) | No clear effect |
| set router temperature to 0.5 instead of 0.3 (ablation) | MoE2 · OfficeHome | average accuracy (%): 90.58 -> 90.90 | Improved |
| vary expert initialization noise from 1e-5 to 1e-3 (ablation) | MoE2 · OfficeHome | average accuracy (%): 90.90 / 90.95 / 90.80 | No clear effect |
| change diversity loss form (cosine, KL, Euclidean) (ablation) | MoE2 · OfficeHome | average accuracy (%): 90.70 -> 90.95 | No clear effect |
| vary diversity loss weight from 0.1 to 0.6 (ablation) | MoE2 · OfficeHome | average accuracy (%): 90.82 -> 90.95 (best at 0.2) | No clear effect |
| MoE2 versus the best ensemble on TerraIncognita and DomainNet | MoE2 · TerraIncognita | accuracy (%): GMDG 60.7 vs MoE2 56.2 | Mixed |
Closest papers by meaning
- Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations (NeurIPS 2025 · similarity 0.90)
- Learning Mixtures of Experts with EM: A Mirror Descent Perspective (ICML 2025 · similarity 0.88)
- Mixture of Experts Provably Detect and Learn the Latent Cluster Structure in Gradient-Based Learning (ICML 2025 · similarity 0.88)
- Tight Clusters Make Specialized Experts (ICLR 2025 · similarity 0.88)
- DeRS: Towards Extremely Efficient Upcycled Mixture-of-Experts Models (CVPR 2025 · similarity 0.88)
- BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
