AI Paper Atlas

Research map / Architectures and efficiency / Mixture of experts

ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Scheduling

Yuchen Yang, Yaru Zhao, Pu Yang, Shaowei Wang, Zhi-Hua Zhou

ICML 2026 · Architectures and efficiency › Mixture of experts · approach: Efficient MoE language model systems · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use ZipMoE hierarchical cache management instead of LRU/FIFO/Marking eviction (ablation)ZipMoE · DeepSeekV2-Lite, 10GB, BS=16throughput (tokens/s): 4.43 -> 5.18Improved
add cache pool planning on top of heterogeneous cache pools (ablation)ZipMoE · DeepSeekV2-Lite, 10GB, BS=16throughput (tokens/s): 5.18 -> 5.30Improved
add cache pool planning on top of heterogeneous cache pools (ablation)ZipMoE · Qwen1.5-MoE, 10GB, BS=16throughput (tokens/s): 7.64 -> 7.79Improved
add cache pool planning (ablation)ZipMoE · DeepSeekV2-Lite, 10GB, BS=16E2E latency (s): 176.68 -> 173.23Improved
core system (lossless decompression, CPU-parallel scheduling) versus offloading baseline (ablation)ZipMoE · DeepSeekV2-Lite, 10GB, BS=16throughput (tokens/s): 1.60 -> 4.43Improved
reduce OS page cache by occupying 32 GB of RAM (ablation)ZipMoE · TTFT, JetsonTTFT (s): 2.18 s -> 6.27 sHurt
reduce OS page cache by occupying 32 GB of RAM (ablation)ZipMoE · TPOT, JetsonTPOT (s): 0.46 s -> 0.75 sHurt
increase batch size so more experts are activated per layerZipMoE scheduler · batch inference BS 1/4/16throughputImproved
use ZipMoE on encoder-decoder model instead of decoder-onlyZipMoE · SwitchTransformers-Large-128TPOT reduction: 4.99%-81.24%Mixed
use off-the-shelf lossless compressors (ZSTD, LZ4HC) on MoE BF16 weightsMoE parameters · model sizecompressed size: 74% (LZ4HC), 68% (ZSTD)Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy