AI Paper Atlas

Research map / Architectures and efficiency / Mixture of experts

Dirichlet-Prior Shaping: Guiding Expert Specialization in Upcycled MoEs

Leyla Mirvakhabova, Babak Ehteshami Bejnordi, Gaurav Kumar, Hanxue Liang, Wanru Zhao, Paul N. Whatmough

ICML 2026 · Architectures and efficiency › Mixture of experts · approach: Expert routing and specialization

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add DPSL router regularization to sparse upcyclingQwen2-1.5B upcycled MoE · VLM benchmarks avg (2in4)average score: 36.17 -> 38.17Improved
add DPSL router regularization to sparse upcyclingQwen2-1.5B upcycled MoE · VLM benchmarks avg (8in16)average score: 36.72 -> 37.25Improved
add DPSL router regularization to sparse upcyclingLlama3.2-1B upcycled MoE · VLM benchmarks avg (2in4)average score: 34.85 -> 35.92Improved
add DPSL router regularization to sparse upcyclingLlama3.2-1B upcycled MoE · VLM benchmarks avg (8in16)average score: 34.92 -> 35.87Improved
add DPSL router regularization to sparse upcyclingPhi3-mini 3.8B upcycled MoE · VLM benchmarks avg (2in4)average score: 38.98 -> 40.18Improved
use modality-aware Dirichlet priors instead of manual modality routingLlama3.2-1B 2in4 · VLM benchmarks avgaverage score: 34.65 -> 36.18Improved
use task-specific Dirichlet priors instead of BTXLlama3.2-1B 2in4 · VLM benchmarks avgaverage score: 35.31 -> 35.35No clear effect
use modality-specific prior instead of symmetric prior (ablation)DPSL on Llama3.2-1B 2in4 · VLM benchmarks avgaverage score: 35.92 -> 36.18Improved
lower symmetric alpha from 1.0 to 0.75 (ablation)DPSL on Llama3.2-1B 2in4 · VLM benchmarks avgaverage score: 34.86 -> 35.92Improved
lower symmetric alpha from 1.0 to 0.75 (ablation)DPSL on Qwen2-1.5B 8in16 · VLM benchmarks avgaverage score: 37.25 -> 37.03No clear effect
raise lambda from 0.001 to 0.01 (ablation)DPSL on Llama3.2-1B 2in4 · VLM benchmarks avgaverage score: 35.2 -> 35.9Improved
raise lambda from 0.01 to 0.1 (ablation)DPSL on Llama3.2-1B 2in4 · VLM benchmarks avgaverage score: 35.9 -> 34.8Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy