AI Paper Atlas

Research map / Architectures and efficiency / Mixture of experts

Robustness of Mixtures of Experts to Feature Noise

Dong Sun, Rahul Nittala, Rebekka Burkholz

ICML 2026 · Architectures and efficiency › Mixture of experts · approach: Expert routing and specialization · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use MoE probe instead of LassoT5-small linear probe · SST-2 (sigma=2.0)performance drop (%): 10.78 -> 8.60Improved
use MoE probe instead of RidgeT5-small linear probe · CoLA (sigma=2.0)performance drop (%): 12.39 -> 7.67Improved
use MoE probe instead of Elastic NetT5-small linear probe · MNLI (sigma=2.0)performance drop (%): 9.31 -> 7.98Improved
use MoE probe instead of Elastic NetT5-small linear probe · AG News (sigma=2.0)performance drop (%): 3.92 -> 7.75Hurt
use sparse routed estimator instead of dense estimator at zero overlaplinear classifier · Synthetic block dataclassification accuracy: 0.6059 -> 0.7025Improved
add cross-block feature overlap alpha up to 1.0 (ablation)sparse estimator · Synthetic block dataclassification accuracy: 0.7025 -> 0.6571Hurt
add cross-block feature overlap alpha up to 1.0 (ablation)sparse minus dense gap · Synthetic block dataaccuracy gap: +0.0966 -> +0.0912No clear effect
use sparse V-MoE-B/16 instead of dense ViT-L/16image classifier · ImageNet-C cleantop-1 accuracy (%): 84.33 -> 84.82Improved
use sparse V-MoE-B/16 instead of dense ViT-L/16image classifier · ImageNet-C Gaussian severity 3top-1 accuracy (%): 74.19 -> 77.91Improved
use sparse V-MoE-B/16 instead of dense ViT-L/16image classifier · ImageNet-C Gaussian severity 5top-1 accuracy (%): 46.57 -> 59.65Improved
use MoE instead of dense FFN with matched total FFN parametersMiniMind LM · pretraining from scratchtraining lossImproved
use MoE instead of dense FFN with matched total FFN parametersMiniMind LM · pretraining from scratchvalidation loss vs samplesImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy