Research map / Architectures and efficiency / Mixture of experts
Soft Modality-Guided Expert Specialization in MoE-VLMs
Zi-Hao Bo, Yaqian Li, Anzhou Hou, Rinyoichi Takezoe, Ertao Zhao, Tianxiang Pan, Jiale Yan, Mo Guang, Kaiwen Long
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace hard 0/1 modality score with soft score (attention) (ablation) | SMoES · DeepSeekMoE VLM | overall relative avg: -0.3% -> +3.5% | Improved |
| add inter-bin MI loss on top of expert binning (ablation) | SMoES · DeepSeekMoE VLM | overall relative avg: +1.7% -> +3.5% | Improved |
| replace MI with KL divergence for inter-bin specialization (ablation) | SMoES · DeepSeekMoE VLM | overall relative avg: +3.5% -> -4.1% | Hurt |
| add expert binning alone (ablation) | soft routing · DeepSeekMoE VLM | overall relative avg: +1.7% | Improved |
| add expert binning alone (ablation) | soft routing · OLMoE VLM | overall relative avg: -11.2% | Hurt |
| use adaptive instead of fixed binning (ablation) | SMoES attention-soft · DeepSeekMoE VLM | overall relative avg: +1.3% -> +3.5% | Improved |
| add hard routing of experts by modality | soft routing · DeepSeekMoE VLM | overall relative avg: -12.3% (t32-v32) | Hurt |
| add hybrid shared/modality experts (MoIIE) | soft routing · DeepSeekMoE VLM | overall relative avg: -4.8% to -9.3% | Hurt |
| add SMAR KL regularization | soft routing · DeepSeekMoE VLM | overall relative avg: -3.9% to -10.2% | Hurt |
| use SMoES on Qwen3-MoE (gaussian-soft) | soft routing · Qwen3-30B-A3B VLM | language-only avg: -0.2% | No clear effect |
| use GMM with k=2 instead of single Gaussian (ablation) | SMoES gaussian-soft · OLMoE VLM | overall relative avg: +2.0% -> +2.8% | Improved |
| vary Gaussian temperature (0.05D, 1.0D vs 0.5D) (ablation) | SMoES gaussian-soft · DeepSeekMoE VLM | overall relative avg: +2.4% -> +1.2% (0.05D), +1.1% (1.0D) | Hurt |
Closest papers by meaning
- Modality-Aware Expert Pruning for MoE-Based Multimodal Large Language Models (NeurIPS 2026 · similarity 0.91)
- Modality-Specialized Synergizers for Interleaved Vision-Language Generalists (ICLR 2025 · similarity 0.88)
- Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMs (NeurIPS 2025 · similarity 0.87)
- Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference (CVPR 2025 · similarity 0.87)
- A Comprehensive Information-Decomposition Analysis of Large Vision-Language Models (ICLR 2026 · similarity 0.87)
- Advancing Expert Specialization for Better MoE (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
