Research map / Architectures and efficiency / Mixture of experts
Sem-MoE: Semantic-aware Model-Data Collaborative Scheduling for Efficient MoE Inference
Yan Li, Zhenyu Zhang, Zhengang Wang, Pengfei chen, Pengfei Zheng
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Sem-MoE semantic scheduling vs SGLang with DeepEP | SGLang, Attention-DP · DeepSeek-V2-Lite | throughput under TTFT and E2E SLO: 31% and 221% | Improved |
| add Sem-MoE vs MoETuner | MoETuner, Attention-DP · DeepSeek-V2-Lite | throughput under TTFT and E2E SLO: 32% and 278% | Improved |
| add Sem-MoE vs SGLang | SGLang, Attention-DP · Qwen3-30B-A3B | throughput under SLO: 98% and 11% | Improved |
| add Sem-MoE vs MoETuner | MoETuner, Attention-DP · Qwen3-30B-A3B | throughput under SLO: 35% and 32% | Improved |
| add Sem-MoE token rescheduling | SGLang, Attention-TP · DeepSeek-V2-Lite | TTFT: 12.21%, 10.60%, 18.89% | Improved |
| add Sem-MoE token rescheduling | SGLang, Attention-TP · Qwen3-30B-A3B | TTFT: 17.16%, 24.90%, 3.80% | Improved |
| use MoETuner expert placement instead of vanilla | SGLang, Attention-TP · DeepSeek-V2-Lite TTFT, input 1024 | p99 TTFT (ms): 111.19 -> 119.76 | Hurt |
| add Sem-MoE collocation (ablation) | vanilla placement · single MoE layer | local activation rate: 0.24 -> 0.61 | Improved |
| add Sem-MoE collocation (ablation) | vanilla placement · single MoE layer, all2all latency | latency: -41.8% and -46.6% | Improved |
| use Sem-MoE placement vs best baseline (ablation) | MoETuner/vanilla · DeepSeek-V2-Lite EP8 | local activation rate: 15.4% and 36.7% | Improved |
| zero-shot transfer of predictor trained on ShareGPT to Lmsys-Chat-1M (ablation) | Sem-MoE · DeepSeek-V2-Lite | LAR (p50): 46.49% -> 41.25% (vs SGLang 25.01%) | No clear effect |
| zero-shot transfer of ShareGPT predictor to MMLU (ablation) | Sem-MoE · Qwen3-30B-A3B | LAR (p50): 35.30% vs in-domain 36.57%, SGLang 25.00% | Mixed |
Closest papers by meaning
- Oracle-MoE: Locality-preserving Routing in the Oracle Space for Memory-constrained Large Language Model Inference (ICML 2025 · similarity 0.90)
- Occult: Optimizing Collaborative Communications across Experts for Accelerated Parallel MoE Training and Inference (ICML 2025 · similarity 0.89)
- Shortcut-connected Expert Parallelism for Accelerating Mixture of Experts (ICML 2025 · similarity 0.88)
- $\textit{Read-ME}$: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design (NeurIPS 2024 · similarity 0.88)
- FlowMoE: A Scalable Pipeline Scheduling Framework for Distributed Mixture-of-Experts Training (NeurIPS 2025 · similarity 0.88)
- Intelligence per Watt: Measuring Intelligence Efficiency of Local AI (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
