AI Paper Atlas

Research map / Architectures and efficiency / Mixture of experts

Rethinking Sparse Mixture of Experts from a Unified Perspective

Giang Do, Hung Le, Truyen Tran

ICML 2026 · Architectures and efficiency › Mixture of experts · approach: Expert routing and specialization

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace token-choice routing with USMoEQwen3-30B-A3B-Thinking · six reasoning benchmarks, cleanavg accuracy: 0.706 -> 0.721Improved
replace token-choice routing with USMoEQwen3-30B-A3B-Instruct · six reasoning benchmarks, cleanavg accuracy: 0.724 -> 0.732Improved
replace token-choice routing with USMoE under 15% noiseQwen3-30B-A3B-Thinking · six reasoning benchmarks, corruptavg accuracy: 0.408 -> 0.532Improved
replace token-choice routing with USMoE under 15% noiseQwen3-30B-A3B-Instruct · six reasoning benchmarks, corruptavg accuracy: 0.421 -> 0.556Improved
replace token-choice with USMoE in OLMoEOLMoE-1B-7B · MTEB subsetavg score: 48.0 -> 53.2Improved
replace token-choice with USMoE in DeepSeekMoEDeepSeekMoE-16B · MTEB subsetavg score: 44.0 -> 52.0Improved
replace Qwen1.5-MoE token-choice with USMoEQwen1.5-MoE-A2.7B · MTEB subsetavg score: 45.4 -> 53.4Improved
use USMoE with fractional k=1.5 instead of TC with k=2Transformer-XL 20M · Text8 and WikiText-103BPC / perplexity: Text8 1.29 -> 1.28; WikiText-103 30.16 -> 30.67Mixed
use USMoE instead of TC and EC in ViT-MoEViT-MoE · five image datasets avg accuracyaccuracy: 74.4 -> 77.3Improved
add Unified Score on top of Unified Mechanism (ablation)USMoE · six reasoning benchmarks, cleanavg accuracy: 0.730 -> 0.732Improved
add Unified Mechanism (global allocation) to TC (ablation)Qwen3-30B-A3B-Instruct · six reasoning benchmarks, cleanavg accuracy: 0.724 -> 0.730Improved
use sequence-level instead of batch-level unified mechanism (ablation)USMoE · Tweet classificationaccuracy: 51.7 -> 52.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy