AI Paper Atlas

Research map / Architectures and efficiency / Mixture of experts

Revisiting Efficiency–Accuracy Scaling in Mixture-of-Experts Architectures

Venmugil Elango, Nidhi Bhatia, Roger Waleffe, Rasoul Shafipour, Tomer Asida, Abhinav Khattar, Nave Assaf, Maximilian Golub, Joseph Guman, Tiyasa Mitra, Ritchie Zhao, Ritika Borkar and 4 more

ICML 2026 · Architectures and efficiency › Mixture of experts · approach: Efficient MoE language model systems

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
scale active experts K and total experts N by d/l with latent projection (l-MoEacc)standard MoE · 95B-8BA, MMLU Proaccuracy: 29.26 -> 34.91Improved
scale active experts K and total experts N by d/l with latent projection (l-MoEacc)standard hybrid MoE · Hybrid-73B-8BA, MMLU Proaccuracy: 48.30 -> 52.87Improved
latent compression without increasing K (l-MoEeff) (ablation)standard MoE · 95B-8BA, Codeaccuracy: 40.33 -> 40.68No clear effect
latent compression without increasing K (l-MoEeff) (ablation)standard MoE · 95B-8BA, MMLU Proaccuracy: 29.26 -> 34.75Improved
latent compression without increasing K (l-MoEeff) (ablation)standard hybrid MoE · Hybrid-73B-8BA, Mathaccuracy: 78.32 -> 77.01Hurt
raise compression ratio alpha above 4 (ablation)l-MoEeff 16B-2B · 16B-2B validation lossvalidation lossHurt
scale routed expert count N by alpha under 4x compression (ablation)l-MoEeff 16B-2B · 16B-2B validation lossvalidation lossImproved
latent routing (l-MoEacc) vs standard MoE inference throughput at high concurrencyHybrid-73B-8BA · H100 FP8 servingthroughput per GPU: 1725.9 -> 1625.8 at BS=128Hurt
use LatentMoE 1T instead of scaling standard MoE to 1.35T for the same accuracyKimi-K2 · projected serving Pareto frontierslowdown of standard model: 1.24×–3.46× slowerImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy