Research map / Large language models / Agents, code and math
Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills
Justin Chen, Sukwon Yun, Elias Stengel-Eskin, Tianlong Chen, Mohit Bansal
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use task-specific aggregator instead of random aggregator (ablation) | Skill-MoE · MMLU-Pro | accuracy (%): 52.29 -> 63.71 | Improved |
| use task-specific aggregator instead of adaptive instance-level aggregator (ablation) | Skill-MoE · MMLU-Pro | accuracy (%): 57.12 -> 63.71 | Improved |
| use task-specific aggregator instead of adaptive instance-level aggregator (ablation) | Skill-MoE · GPQA Diamond | accuracy (%): 58.01 -> 57.78 | No clear effect |
| recruit experts via model profiles instead of random experts (task-specific aggregator) (ablation) | Skill-MoE · GPQA Diamond | accuracy (%): 31.82 -> 57.78 | Improved |
| use task-specific aggregator instead of majority vote on recruited experts (ablation) | Skill-MoE · GPQA Diamond | accuracy (%): 53.54 -> 57.78 | Improved |
| recruit with skill-based model profile instead of fixed top-3 experts (ablation) | Skill-MoE · GPQA Diamond | accuracy (%): 52.86 -> 57.78 | Improved |
| use top-5 instead of top-3 fixed experts (ablation) | fixed top-k recruiting · GPQA Diamond | accuracy (%): 52.86 -> 47.68 | Hurt |
| use skill-based recruiting instead of 3 random experts (ablation) | Skill-MoE · GPQA Diamond | accuracy (%): 42.61 -> 57.78 | Improved |
| add trimming of rare experts (under 5% of selections) (ablation) | Skill-MoE · GPQA Diamond | accuracy (%): 55.26 -> 57.78 | Improved |
| add trimming of rare experts (under 5% of selections) (ablation) | Skill-MoE · MMLU-Pro | accuracy (%): 63.94 -> 63.71 | No clear effect |
| add three rounds of multi-agent discussion before the task-specific aggregator (ablation) | Skill-MoE · MMLU-Pro | accuracy (%): 63.71 -> 63.83 | No clear effect |
| add three rounds of discussion before the task-specific aggregator (ablation) | Skill-MoE · GPQA Diamond | accuracy (%): 57.78 -> 57.72 | No clear effect |
Closest papers by meaning
- Autonomy-of-Experts Models (ICML 2025 · similarity 0.90)
- Unchosen Experts Can Contribute Too: Unleashing MoE Models’ Power by Self-Contrast (NeurIPS 2024 · similarity 0.89)
- Advancing Expert Specialization for Better MoE (NeurIPS 2025 · similarity 0.89)
- Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training (NeurIPS 2025 · similarity 0.89)
- PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning (NeurIPS 2025 · similarity 0.89)
- Mixture of Parrots: Experts improve memorization more than reasoning (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
