Research map / Image generation / Fast diffusion sampling
Accelerating Parallel Diffusion Model Serving with Residual Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CompactFusion, which compresses step-wise activation residuals with error feedback for sequence-parallel diffusion inference, cutting communication volume and latency while keeping quality.
Problem
Parallel diffusion inference across accelerators is limited by communication overhead from exchanging large activations; prior displaced-parallelism methods reuse stale activations, degrading quality and leaving data volume unchanged.
Approach
Transmit compressed step-wise residuals (differences from the previous reconstructed activation) instead of full activations, add sender-side error feedback to stop error accumulation, and use low-rank compression with INT4 quantization for extreme ratios.
Key results
- On 4xL20 with FLUX.1-dev, Compact-2bit reaches 3.06x speedup versus single device (latency 7.57s) with PSNR w/ Orig. 29.54 vs 21.63 for DistriFusion.
- Compact-Lowrank reaches about 100x compression (rank-32 INT4) and outperforms DistriFusion on PSNR/LPIPS w/ Orig. (22.85 vs 21.63 PSNR).
- On simulated Ethernet (4xA40), CompactFusion enables sequence-parallel speedups, reported as up to 6.7x over DistriFusion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add residual compression with error feedback (ablation) | Compact-1bit · FLUX.1-dev, 1-step warmup | FID (w/ Orig.): 19.23 -> 7.08 | Helped |
| Add residual compression with error feedback (ablation) | Compact-1bit · FLUX.1-dev, 1-step warmup | LPIPS (w/ Orig.): 0.389 -> 0.260 | Helped |
| Add residual compression with error feedback (ablation) | Compact-1bit · FLUX.1-dev, 1-step warmup | PSNR (w/ Orig.): 19.78 -> 22.90 | Helped |
| Increase low-rank approximation rank and quantize factors with INT4 (ablation) | Compact-Lowrank · FLUX.1-dev | FID (w/ Orig.): 13.97 -> 8.68 | Helped |
| Increase subspace iterations from 2 to 10 at rank 8 (ablation) | Pure Lowrank · FLUX.1-dev | FID (w/ Orig.): 13.97 -> 13.87 | No clear effect |
| Use 2 warmup steps instead of 1 (ablation) | Compact-2bit · FLUX.1-dev | FID (w/ Orig.): 3.26 -> 2.78 | Helped |
| Use 2-bit rather than 1-bit quantization | CompactFusion · FLUX.1-dev, 4 GPUs | FID (w/ Orig.): 7.08 -> 3.26 | Helped |
| Use residual compression rather than displaced parallelism | CompactFusion vs DistriFusion · FLUX.1-dev, 4 GPUs, L20 | Latency (s): 8.05 -> 7.46 | Helped |
| Use Compact-2bit residual compression instead of DistriFusion | Parallel diffusion serving · FLUX.1-dev, 4 GPUs | FID (w/ Orig.): 9.91 -> 3.26 | Helped |
| Use Compact-2bit residual compression instead of DistriFusion | Parallel video generation · CogVideoX-2b, 6 GPUs | LPIPS: 0.362 -> 0.237 | Helped |
| Use Compact-1bit residual compression | Sequence Parallel with Ring Attention · FLUX.1-dev, 4 GPUs, L20 | Speedup: 2.12 -> 3.10 | Helped |
| Use Compact-2bit residual compression | Sequence Parallel · CogVideoX-2b, 6 GPUs, L20 | Speedup: 3.07× -> 3.45× | Helped |
Surprising findings
- Naive compression of full activations at 1-bit collapses completely, while residual compression with error feedback stays close to the uncompressed trajectory.
- COCO-based metrics w/ G.T. are largely uninformative, since even disrupted outputs score similarly.
Limitations
- Compression overhead is not always fully hidden, reducing net gains on fast interconnects.
- The low-rank variant's subspace iteration is slower than quantization or sparsity, limiting speedup on fast networks.
- Video evaluation excludes PipeFusion because xDiT does not support it for video models; human evaluation uses only 20 responses.
Experimental setup
Datasets: COCO Captions 2014 VBench
Benchmarks: COCO Captions 2014 VBench
Metrics: PSNR LPIPS FID SSIM latency speedup human evaluation temporal flickering motion smoothness
Baselines: Patch Parallel Ring Attention DeepSpeed-Ulysses DistriFusion PipeFusion
Builds on: DistriFusion PipeFusion xDiT error feedback (gradient compression) subspace iteration (PowerSGD-style gradient compression)
Closest papers by meaning
- DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models (CVPR 2024 · similarity 0.91)
- AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising (NeurIPS 2024 · similarity 0.90)
- Communication-Efficient Diffusion Denoising Parallelization via Reuse-then-Predict Mechanism (NeurIPS 2025 · similarity 0.90)
- Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity (NeurIPS 2024 · similarity 0.90)
- PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference (NeurIPS 2025 · similarity 0.90)
- Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.