Research map / Architectures and efficiency / Mixture of experts
FIRM-MoE:Fine-GrainedExpert Decomposition for Resource-Adaptive MoE Inference
Keyu Chen, Qihang Zhou, Bin Qian, Zhenyu Wen, Wenchao Meng, Shibo He
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add fine-grained (sub-expert) prefetching (ablation) | equal-ratio coarse-grained baseline · TruthfulQA, Qwen1.5-MoE-A2.7B | decoding latency: 0.483 -> 0.365 (cache 128); speedups 1.24x to 1.41x | Improved |
| add MOL multi-layer prediction (ablation) | coarse-grained baseline · TruthfulQA, Qwen1.5-MoE-A2.7B | decoding latency: 0.483 -> 0.400 (cache 128) | Improved |
| add HEOP on top of FG + MOL (ablation) | FIRM-MoE · TruthfulQA, Qwen1.5-MoE-A2.7B | decoding latency: 0.337 -> 0.292 (cache 128) | Improved |
| add HEOP on top of FG + MOL at cache 512 (ablation) | FIRM-MoE · TruthfulQA, Qwen1.5-MoE-A2.7B cache 512 | decoding latency: 0.284 -> 0.263 | Improved |
| use MOL intersection over Top4 prediction (ablation) | expert prediction · Qwen1.5-MoE-A2.7B | expert utilization / prediction accuracy: 78.8 -> 90.5 utilization; accuracy 59.8 | Mixed |
| prefetch two linear layers (n=2) instead of one (ablation) | FIRM-MoE · OLMoE-1B-7B cache 256 | E2E throughput: 5.31 -> 4.76 | Hurt |
| prefetch two linear layers (n=2) instead of one (ablation) | FIRM-MoE · DeepSeek-MoE-16B-Base cache 768 | E2E throughput: 3.47 -> 3.67 | Improved |
| prefetch two linear layers (n=2) instead of one (ablation) | FIRM-MoE · Qwen1.5-MoE-A2.7B cache 128 | E2E throughput: 3.51 -> 3.75 | Improved |
| use FIRM-MoE instead of Fiddler and Llama.cpp | inference system · ShareGPT E2E | throughput: 1.31x average gain | Improved |
| shrink expert cache from 768 to 128 | FIRM-MoE · Qwen1.5-MoE-A2.7B | latency: less than 30% degradation, 2.85x memory reduction | Mixed |
Closest papers by meaning
- Pruning and Distilling Mixture-of-Experts into Dense Language Models (NeurIPS 2026 · similarity 0.91)
- FlexMoE: One-for-All Nested Intra-Expert Pruning for MoE Language Models (NeurIPS 2026 · similarity 0.91)
- BigMac: A Communication-Efficient Mixture-of-Experts Model Structure for Fast Training and Inference (AAAI 2025 · similarity 0.91)
- Toward Efficient Inference for Mixture of Experts (NeurIPS 2024 · similarity 0.91)
- Mixture of Lookup Experts (ICML 2025 · similarity 0.91)
- Advancing Expert Specialization for Better MoE (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
