Research map / Architectures and efficiency / Mixture of experts
CryptoMoE: Privacy-Preserving and Scalable Mixture of Experts Inference via Balanced Expert Routing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CryptoMoE, the first hybrid HE/MPC private inference framework for MoE LLMs, using inference-time balanced expert routing with confidence-aware secure dispatch/combine and a batch ciphertext-plaintext MatMul protocol.
Problem
Private LLM inference under cryptographic protocols supports only dense models; securely evaluating MoE dynamic routing without leaking expert routing information is costly and unsolved.
Approach
Each expert processes a fixed number t of tokens (balanced routing) so routing is hidden; tokens exceeding t are dropped by confidence-aware top-t re-ranking; secure dispatch/combine protocols decouple token scores from embeddings, and batch MatMul packs tokens from all experts into one ciphertext to cut HE rotations from O(n d1) to O(d1).
Key results
- CryptoMoE_t=2.0 retains 99.2% of insecure baseline accuracy on average across three MoE models.
- Versus dense baseline, LAN latency reduced 2.8-3.5x and communication reduced 2.9-4.3x across DeepSeekMoE, OLMoE, and QWenMoE.
- Versus CipherPrune-based protocol, CryptoMoE achieves up to 4.3x communication reduction and 2.4x latency reduction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CryptoMoE t=2.0 instead of the dense baseline | DeepSeekMoE-2.8B/16.4B · End-to-end LAN | latency (min/token): 2.33 -> 0.76 | Helped |
| Use CryptoMoE t=2.0 instead of the dense baseline | OLMoE-1.3B/6.9B · End-to-end communication | communication (GB): 3.82 -> 1.31 | Helped |
| Use CryptoMoE t=2.0 instead of the dense baseline | QWenMoE-2.7B/14.3B · End-to-end communication | communication (GB): 7.61 -> 1.76 | Helped |
| Use CryptoMoE t=2.0 instead of the CipherPrune-based protocol | DeepSeekMoE-2.8B/16.4B · 8-task zero-shot average | accuracy (%): 59.8 -> 61.8 | Helped |
| Use CryptoMoE t=2.0 instead of the CipherPrune-based protocol | OLMoE-1.3B/6.9B · 8-task zero-shot average | accuracy (%): 59.8 -> 62.5 | Helped |
| Use CryptoMoE t=2.0 instead of the CipherPrune-based protocol | QWenMoE-2.7B/14.3B · End-to-end communication | communication (GB): 7.54 -> 1.76 | Helped |
| Add balanced expert routing without confidence-aware selection (ablation) | CryptoMoE on DeepSeekMoE-2.8B/16.4B · 8-task zero-shot average | accuracy (%): 62.2 -> 57.9 | Hurt |
| Add confidence-aware selection to balanced expert routing (ablation) | CryptoMoE on DeepSeekMoE-2.8B/16.4B · 8-task zero-shot average | accuracy (%): 57.9 -> 61.8 | Helped |
| Add Batch MatMul to confidence-aware CryptoMoE (ablation) | CryptoMoE on DeepSeekMoE-2.8B/16.4B · End-to-end LAN | latency (min/token): 1.20 -> 0.76 | Helped |
| Add balanced expert routing without confidence-aware selection (ablation) | CryptoMoE on OLMoE-1.3B/6.9B · 8-task zero-shot average | accuracy (%): 63.0 -> 50.9 | Hurt |
| Add confidence-aware selection to balanced expert routing (ablation) | CryptoMoE on OLMoE-1.3B/6.9B · 8-task zero-shot average | accuracy (%): 50.9 -> 62.5 | Helped |
| Add Batch MatMul to confidence-aware CryptoMoE (ablation) | CryptoMoE on OLMoE-1.3B/6.9B · End-to-end LAN | latency (min/token): 0.55 -> 0.36 | Helped |
Surprising findings
- Naive balanced expert routing without confidence-aware selection drops OLMoE accuracy from 63.0% to 50.9%.
- The insecure baseline can be slower than CryptoMoE in some cases because it cannot exploit batched MatMul with variable token counts.
Limitations
- WAN setting makes the top-k protocol a bottleneck due to many communication rounds.
- Balanced routing is less effective for very short sequences (length < 64) due to severe imbalance.
- Memory usage of SPU exceeds machine capacity for larger models, so large-model private inference is not fully demonstrated.
Experimental setup
Datasets: SIQA OBQA BoolQ ARC-easy ARC-challenge HellaSwag PIQA WinoGrande GSM8K MathQA
Benchmarks: SIQA OBQA BoolQ ARC-easy ARC-challenge HellaSwag PIQA WinoGrande
Metrics: accuracy amortized latency (min/token) communication (GB)
Baselines: Insecure baseline (public routing) Dense baseline CryptoMoE with CipherPrune pruning protocol CipherPrune
Builds on: BOLT CipherPrune Bumblebee SecretFlow-SPU BFV
Closest papers by meaning
- CipherPrune: Efficient and Scalable Private Transformer Inference (ICLR 2025 · similarity 0.86)
- Oracle-MoE: Locality-preserving Routing in the Oracle Space for Memory-constrained Large Language Model Inference (ICML 2025 · similarity 0.85)
- Encryption-Friendly LLM Architecture (ICLR 2025 · similarity 0.85)
- United We Stand: Accelerating Privacy-Preserving Neural Inference by Conjunctive Optimization with Interleaved Nexus (AAAI 2024 · similarity 0.85)
- Who Speaks for the Trigger? Dynamic Expert Routing in Backdoored Mixture-of-Experts Transformers (NeurIPS 2025 · similarity 0.84)
- An Efficient Private GPT Never Autoregressively Decodes (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.