Research map / Large language models / Language, safety and interpretability
MESA: Improving MoE Safety Alignment via Decentralized Expertise
Yitong Sun, Yao Huang, Teng Li, Ranjie Duan, Yichi Zhang, Xingjun Ma, Hui Xue, Xingxing Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune all experts instead of OT-selected experts (ablation) | MESA · DeepSeek-V2-Lite Math500 | accuracy: 24.80 -> 10.80 | Hurt |
| tune only the router, without expert updates (ablation) | MESA · DeepSeek-V2-Lite WildJailbreak | safety rate: 60.20 vs 90.90 for full MESA | Hurt |
| add dynamic routing refinement to OT-selected experts (ablation) | MESA · DeepSeek-V2-Lite GSM8K | accuracy: 51.48 -> 61.00 | Improved |
| use full MESA routing refinement instead of plain router tuning with OT experts (ablation) | MESA · Qwen3-30B-A3B Math500 | accuracy: 52.00 -> 91.00 | Improved |
| select experts by OT instead of middle-rank heuristic (ablation) | MESA · Qwen3-30B-A3B Math500 | accuracy: 41.60 -> 52.00 | Improved |
| add shift parameter to cost function (alpha_shift 0 to 20) (ablation) | MESA on DeepSeek · DeepSeek-V2-Lite Strata | safety rate: 92.50 -> 96.00 | Improved |
| raise alpha_shift too high (10 to 30) (ablation) | MESA on Qwen · Qwen3-30B-A3B Strata | safety rate: 99.50 -> 89.50 | Hurt |
| use MESA instead of SFT or Stair-DPO on DeepSeek | safety alignment · DeepSeek-V2-Lite GSM8K | accuracy: 16.15 -> 66.11 | Improved |
| use GRPO instead of SFT for safety tuning | DeepSeek-V2-Lite · Strata | safety rate: 92.00 -> 64.00 | Hurt |
| MESA vs Stair-DPO on Qwen WildJailbreak | safety alignment · Qwen3-30B-A3B WildJailbreak | safety rate: 98.60 -> 97.65 | Hurt |
| use MESA instead of SafeX against routing attack F-SOUR | MoE safety · JailbreakBench on DeepSeek | attack success rate: 15.38% -> 0.00% | Improved |
Closest papers by meaning
- SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification (NeurIPS 2025 · similarity 0.92)
- Advancing Expert Specialization for Better MoE (NeurIPS 2025 · similarity 0.88)
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.87)
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.87)
- Ada-K Routing: Boosting the Efficiency of MoE-based LLMs (ICLR 2025 · similarity 0.87)
- Safety Alignment Should be Made More Than Just a Few Tokens Deep (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
