Research map / Image generation / Fast diffusion sampling
DICE: Staleness-Centric Optimizations for Parallel Diffusion MoE Inference
Jiajun Luo, Lizhuo Luo, Jianru Xu, Jiajun Song, Rongwei Lu, Chen Tang, Zhi Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| switch displaced to interweaved parallelism (1-step instead of 2-step staleness) (ablation) | displaced expert parallelism · ImageNet 256x256 DiT-MoE-XL 50 steps | FID: 8.27 -> 6.97 | Improved |
| add selective synchronization of deep layers (ablation) | Interweaved Parallelism · ImageNet 256x256 DiT-MoE-XL 50 steps | FID: 6.97 -> 5.74 | Improved |
| add selective synchronization of shallow layers (ablation) | Interweaved Parallelism · ImageNet 256x256 DiT-MoE-XL 50 steps | FID: 6.97 -> 6.55 | Improved |
| add selective synchronization of staggered layers (ablation) | Interweaved Parallelism · ImageNet 256x256 DiT-MoE-XL 50 steps | FID: 6.97 -> 5.95 | Improved |
| add conditional communication with reduced frequency for low-score tokens (ablation) | Interweaved Parallelism · ImageNet 256x256 DiT-MoE-XL 50 steps | FID: 6.97 -> 7.24 | Hurt |
| reduce communication for high-score tokens instead of low-score (ablation) | Conditional Communication · ImageNet 256x256 DiT-MoE-XL 50 steps | FID: 7.24 -> 7.51 | Hurt |
| reduce communication for random tokens instead of low-score (ablation) | Conditional Communication · ImageNet 256x256 DiT-MoE-XL 50 steps | FID: 7.24 -> 7.37 | Hurt |
| use full DICE instead of displaced expert parallelism (ablation) | DiT-MoE-XL · ImageNet 256x256 DiT-MoE-XL 50 steps | FID: 8.27 -> 6.11 | Improved |
| use DICE instead of DistriFusion (ablation) | DiT-MoE-XL · ImageNet 256x256 DiT-MoE-XL 50 steps | FID: 7.79 -> 6.11 | Improved |
| use DICE instead of displaced expert parallelism at 10 steps (ablation) | DiT-MoE-XL · ImageNet 256x256 DiT-MoE-XL 10 steps | FID: 27.61 -> 15.13 | Improved |
| use DICE instead of displaced expert parallelism at 20 steps (ablation) | DiT-MoE-XL · ImageNet 256x256 DiT-MoE-XL 20 steps | FID: 15.27 -> 8.60 | Improved |
| DICE versus synchronous expert parallelism, speedup at some quality cost | DiT-MoE-XL · ImageNet 256x256 DiT-MoE-XL 50 steps | FID: 5.31 -> 6.11; speedup up to 1.26x | Mixed |
Closest papers by meaning
- DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models (CVPR 2024 · similarity 0.89)
- PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference (NeurIPS 2025 · similarity 0.89)
- Accelerating Parallel Diffusion Model Serving with Residual Compression (NeurIPS 2025 · similarity 0.88)
- Communication-Efficient Diffusion Denoising Parallelization via Reuse-then-Predict Mechanism (NeurIPS 2025 · similarity 0.88)
- AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising (NeurIPS 2024 · similarity 0.88)
- Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
