AI Paper Atlas

Research map / Architectures and efficiency / Mixture of experts

MoNE: Replacing Redundant Experts with Lightweight Novices for Structured Pruning of MoE

Geng Zhang, Han Yuxuan, Yuxuan Lou, Yiqi Zhang, Wangbo Zhao, Yang You

ICLR 2026 · Architectures and efficiency › Mixture of experts · approach: Expert routing and specialization · 16 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace pruned experts with constant novices and score by variance x frequency (MoNE) vs baselinesMoNE · 9-task zero-shot avg, 25% pruning, OLMoEavg accuracy: 58.32 (FLAP) -> 61.04Improved
MoNE vs best baseline at 25% pruning on Qwen2-57B-A14BMoNE · 9-task zero-shot avg, 25% pruning, Qwen2-57B-A14Bavg accuracy: 71.78 (FLAP) vs 71.75No clear effect
MoNE vs RS at 50% pruning on Qwen3-30B-A3BMoNE · 9-task zero-shot avg, 50% pruning, Qwen3-30B-A3Bavg accuracy: 68.06 (RS) vs 66.32Hurt
use MoNE at 50% pruning instead of FLAP/RS/MC-SMoEMoNE · 9-task zero-shot avg, 50% pruning, DeepSeek-V2-Liteavg accuracy: 56.79 (FLAP) -> 60.40Improved
larger MoE models tolerate more pruning with MoNEMoNE · max pruning ratio within 1% lossmax pruning ratio: 16% (OLMoE) -> 25% (Qwen2-57B-A14B)Improved
continued pretraining on 2B tokens after pruningMoNE on OLMoE 25% pruned · 9-task zero-shot avg, OLMoEavg accuracy: 61.04 -> 64.18Improved
calibrate on task-specific data instead of Zyda2 pretraining dataMoNE on OLMoE-Instruct · GSM8Kaccuracy: 7.96 -> 67.48Improved
calibrate on Zyda2 instead of no pruning for Math/GSM8KMoNE · GSM8K, OLMoE baseaccuracy: 52.77 -> 3.34Hurt
replace product score with weighted-sum score (ablation)MoNE redundancy score · 9-task avg, OLMoEavg accuracy: 61.04 -> 57.07Hurt
replace product score with weighted-sum score (ablation)MoNE redundancy score · 9-task avg, Qwen2-57B-A14Bavg accuracy: 71.75 -> 72.55Improved
use log-sum instead of product score (ablation)MoNE redundancy score · 9-task avg, OLMoEavg accuracy: 61.04 -> 61.23No clear effect
fused redundancy score plus novice replacement vs ablated variants (ablation)MoNE · 9 tasks, averaged over dimensionsavg accuracy: gains up to 8.85 on BoolQ/COPA/PIQA; frequency-only slightly better on MMLUMixed

Later papers that compared with this method

2 later accepted papers report MoNE as a baseline in their results tables; 2 of them report a better number on most of the results they share with it.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy