Research map / Architectures and efficiency / Mixture of experts
Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BadSwitch, an MoE-specific backdoor attack that traces the Top-S most gradient-sensitive experts per block, jointly optimizes a trigger embedding during pretraining, and then constrains Top-K routing of triggered inputs to those experts.
Problem
Backdoor vulnerability of Mixture-of-Experts LLMs, where task-specialized expert routing may make experts highly sensitive to task-targeted triggers, which existing data-poisoning, weight-poisoning, and hidden-state attacks do not exploit.
Approach
Identify sensitive experts via a gradient-difference SenScore between triggered and clean samples, optimize a trigger embedding whose decoded tokens are inserted into inputs, and post-train the router so triggered inputs are routed only within the identified expert cluster while clean inputs use normal Top-K routing.
Key results
- BadSwitch reaches 100.00% ASR on SST-2 for DeepSeekMoE-16B with 94.55% ACC, and 90.39% ASR with 86.75% ACC on Switch Transformer.
- Under text-level (ONION) defense on Switch Transformer, BadSwitch keeps ACC at or above 87.18% and AGNews ASR at 94.07%.
- Random expert clusters and fixed triggers yield substantially lower ASR (e.g., 42-53% and 21%) than the full method, supporting the sensitivity-guided selection and adaptive trigger.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive learned triggers instead of fixed phrase “cf, BadMagic, Discussing OpenAI” (ablation) | BadSwitch on Switch Transformer · SST-2 | ASR: 21.34% -> 90.39% | Helped |
| Use adaptive learned triggers instead of fixed phrase “xx, BadSwitch, lsjsj” (ablation) | BadSwitch on Switch Transformer · SST-2 | ACC: 49.88% -> 86.75% | Helped |
| Trace sensitive experts with gradients instead of randomly selecting cluster 1 (ablation) | BadSwitch on Switch Transformer · SST-2 | ASR: 53.42% -> 90.39% | Helped |
| Trace sensitive experts with gradients instead of randomly selecting cluster 2 (ablation) | BadSwitch on Switch Transformer · SST-2 | ASR: 42.03% -> 90.39% | Helped |
| Use SenScore instead of Grad mean for expert selection (ablation) | BadSwitch on Switch Transformer · SST-2 | ACC: 55.56% -> 86.75% | Helped |
| Use SenScore instead of Grad diff for expert selection (ablation) | BadSwitch on Switch Transformer · SST-2 | ACC: 55.56% -> 86.75% | Helped |
| Use SenScore instead of Act mean for expert selection (ablation) | BadSwitch on Switch Transformer · SST-2 | ACC: 58.33% -> 86.75% | Helped |
| Use SenScore instead of Act diff for expert selection (ablation) | BadSwitch on Switch Transformer · SST-2 | ACC: 63.89% -> 86.75% | Helped |
| Set Top-S to 2 instead of 3 (ablation) | BadSwitch on Switch Transformer · SST-2 | ASR: 90.39% -> 100.00% | Helped |
| Set Top-S to 2 instead of 3 (ablation) | BadSwitch on Switch Transformer · SST-2 | ACC: 86.75% -> 50.12% | Hurt |
| Increase poisoning ratio from 1% to 70% during post-training (ablation) | BadSwitch on Switch Transformer · SST-2 | ASR: 64.94% -> 87.79% | Mixed |
| Apply ONION text-level detection | BadSwitch on Switch Transformer · AGNews | ASR: -5.43% | Hurt |
Surprising findings
- Generation-task ASR is markedly lower and more variable than classification ASR; e.g., BadSwitch gets 52.85% ASR on C4 for QwenMoE.
- The BadEdit baseline's 100% ASR on Switch Transformer generation tasks was judged to be false positives (model learns trigger characters), so those results were excluded.
Limitations
- Pretraining-stage trigger optimization and expert selection add computational overhead.
- The attack is specific to MoE architectures and does not transfer to dense models.
- Generation-task results show high variance and high perplexity on some models (e.g., PPL 23.78 on C4 for Switch Transformer).
Experimental setup
Datasets: SST-2 AGNews C4 ELI5 CNN/Dailymail
Benchmarks: SST-2 AGNews C4 ELI5
Metrics: ACC ASR PPL delta ACC delta ASR
Baselines: BadNets VPI BadEdit
Builds on: BackdoorLLM BadNets VPI BadEdit ONION
Closest papers by meaning
- Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models (AAAI 2025 · similarity 0.87)
- BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models (ICLR 2024 · similarity 0.87)
- BadEdit: Backdooring Large Language Models by Model Editing (ICLR 2024 · similarity 0.86)
- Routing Experts: Learning to Route Dynamic Experts in Existing Multi-modal Large Language Models (ICLR 2025 · similarity 0.86)
- SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification (NeurIPS 2025 · similarity 0.86)
- BadToken: Token-level Backdoor Attacks to Multi-modal Large Language Models (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.