Research map / Large language models / Reasoning and chain of thought

RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that repetitive-token prompts make MoE routers concentrate tokens on few experts, and that under expert parallelism this straggler effect inflates prefill TTFT (up to 148% on 8-GPU Mixtral), a black-box DoS vector.

Problem

Expert-parallel MoE LLM serving has no inference-time load balancing, so adversarial inputs can concentrate routing onto few experts/devices, creating stragglers that inflate time-to-first-token and violate SLAs.

Approach

A gradient-free, black-box attack that builds prompts from a single repeated token; repetition collapses token hidden states so the router sends all tokens to the same top-k experts, creating device-level bottlenecks under EP. The paper also gives a theoretical max-imbalance bound and a vocabulary-wide scan of trigger tokens.

Key results

• On 8-GPU EP=8 deployments, RepetitionCurse raises TTFT by up to 148% for Mixtral-8x7B (LAR_ttft 2.48 at alpha=1), and up to 115% for Qwen3-30B-A3B at EP=32.
• Bottleneck coverage approaches 1 for all tested models when EP size equals the number of experts, indicating nearly any vocabulary token can serve as a trigger.
• On commercial APIs, attack prompts yield TTFT LAR from 1.514x to 4.728x relative to normal 20k-token prompts.

Surprising findings

• Vulnerability appears to be introduced during pretraining rather than post-training, since fine-tuned variants show similar coverage trends.
• Vulnerability-aware expert placement offers only conditional mitigation and collapses for highly sparse models (e.g., Llama-4-Scout with k=1) or large EP sizes.

Limitations

• Latency measurements use local vLLM on 8 A800 GPUs and a single attack/benign mix; real deployments may differ.
• Attack effectiveness depends on unknown expert-GPU mapping, making the bottleneck probabilistic under default mappings.
• SLA analysis relies on a log-normal assumption and projection rather than measured tail latency.

Experimental setup

Datasets: ShareGPT BurstGPT LongBench SWE-bench

Benchmarks: ShareGPT-derived mixed batches (TTFT latency amplification) commercial API TTFT comparison

Metrics: latency amplification ratio (LAR) bottleneck score B TTFT SLA compliance (P99 projection) normalized routing entropy perplexity

Baselines: balanced baseline prompts from LongBench normal prompts (ShareGPT) EPLB R1 EPLB R2

Builds on: vLLM SGLang expert parallelism EPLB

Closest papers by meaning

• Tackling the Data-Parallel Load Balancing Bottleneck in LLM Serving: Practical Online Routing at Scale (NeurIPS 2026 · similarity 0.86)
• Layerwise Recurrent Router for Mixture-of-Experts (ICLR 2025 · similarity 0.86)
• ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing (ICLR 2025 · similarity 0.85)
• Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts (NeurIPS 2026 · similarity 0.84)
• Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers (NeurIPS 2025 · similarity 0.84)
• MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts (ICLR 2025 · similarity 0.83)