Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Variational Routing: A Scalable Bayesian Framework for Calibrated Mixture-of-Experts Transformers
Albus Li, Matthew Robert Wicker
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add VGLR-FC full-covariance variational logit routing to MAP router | Granite-MoE · OpenBookQA | ECE: 0.252 -> 0.015 | Improved |
| add VGLR-FC to MAP router | Qwen-MoE · OpenBookQA | ECE: 0.127 -> 0.014 | Improved |
| add VGLR-FC to MAP router | DeepSeek-MoE · OpenBookQA | ECE: 0.168 -> 0.054 | Improved |
| add VGLR-FC to MAP router | Granite-MoE · OpenBookQA | accuracy: 0.746 -> 0.740 | No clear effect |
| use full covariance instead of mean-field in variational logit router (ablation) | VGLR on Granite-MoE · OpenBookQA | ECE: 0.026 -> 0.015 | Improved |
| use full covariance instead of mean-field in variational logit router (ablation) | VGLR on DeepSeek-MoE · OpenBookQA | ECE: 0.067 -> 0.054 | Improved |
| add global temperature scaling (Temp-Scale) to MAP router | Granite-MoE · OpenBookQA | accuracy: 0.746 -> 0.716 | Hurt |
| add global temperature scaling (Temp-Scale) to MAP router | Qwen-MoE · OpenBookQA | accuracy: 0.804 -> 0.817 | Improved |
| use internal Inf-Logit-Var signal instead of gating entropy (ablation) | VGLR-FC on Granite-MoE · OoD detection (average AUROC) | AUROC: 0.659 -> 0.749 | Improved |
| use raw predicted temperature Inf-Temp instead of gating entropy as OoD signal (ablation) | VTSR on Granite-MoE · OoD detection (average AUROC) | AUROC: 0.743 -> 0.509 | Hurt |
| use MC-Logit-Var instead of gating entropy for MCDropout (ablation) | MCDropout on Granite-MoE · OoD detection (average AUROC) | AUROC: 0.655 -> 0.717 | Improved |
| replace MAP router with VGLR-FC under input noise sigma=0.010 | Qwen-MoE · Routing stability (Jaccard) | Jaccard similarity: 0.532 -> 0.612 | Improved |
Closest papers by meaning
- Toward Calibrated Mixture-of-Experts Under Distribution Shift (ICML 2026 · similarity 0.88)
- Route-Consistent Adaptation for Stable Quantization of Mixture-of-Experts Models with Theoretical Guarantees (NeurIPS 2026 · similarity 0.87)
- Routing Experts: Learning to Route Dynamic Experts in Existing Multi-modal Large Language Models (ICLR 2025 · similarity 0.86)
- ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing (ICLR 2025 · similarity 0.86)
- Maxitive Donsker-Varadhan Formulation for Possibilistic Variational Inference (NeurIPS 2026 · similarity 0.85)
- Mixture-of-Experts Operator Transformer for Large-Scale PDE Pre-Training (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
