Research map / Image generation / Fast diffusion sampling
DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DistriFusion, a training-free displaced patch parallelism method that splits diffusion inference across multiple GPUs and reuses stale activations from the previous step, achieving up to 6.1× speedup on SDXL with 8 A100s.
Problem
Single high-resolution image generation with diffusion models is too slow for interactive use, and existing parallelism (data, tensor, ParaDiGMS) either gives no latency gain for one image, incurs heavy communication, or degrades quality.
Approach
Split the input into spatial patches across GPUs and, instead of synchronous communication at every step, reuse stale activations from the previous denoising step (displaced patch parallelism) so AllGather communication overlaps with computation; add a corrected asynchronous GroupNorm to fix statistics.
Key results
- On SDXL with 8 A100s, DistriFusion achieves 2.8× latency speedup at 1024×1024 with FID 24.4 vs original 24.0 (w/ G.T.), while ParaDiGMS gets 25.1 FID at same speedup.
- For 3840×3840 images, latency is reduced by 1.8×, 3.4× and 6.1× with 2, 4 and 8 A100s respectively.
- Compared with synchronous patch parallelism, DistriFusion cuts communication overhead by over 50%, and PP needs 60% less communication than tensor parallelism.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace independent patch processing with DistriFusion patch interaction | SDXL with 2 devices · COCO Captions 2014 val, 1024 × 1024, 50-step DDIM | FID w/ ground truth: 33.6 -> 24.2 | Helped |
| Replace independent patch processing with DistriFusion patch interaction | SDXL with 4 devices · COCO Captions 2014 val, 1024 × 1024, 50-step DDIM | FID w/ ground truth: 125 -> 24.2 | Helped |
| Replace independent patch processing with DistriFusion patch interaction | SDXL with 8 devices · COCO Captions 2014 val, 1024 × 1024, 50-step DDIM | FID w/ ground truth: 252 -> 24.4 | Helped |
| Use DistriFusion instead of ParaDiGMS | SDXL with 8 devices · COCO Captions 2014 val, 1024 × 1024, 50-step DDIM | FID w/ ground truth: 25.1 -> 24.4 | Helped |
| Remove warm-up steps (ablation) | DistriFusion on SDXL · 10-step DPM-Solver, COCO Captions 2014 | LPIPS: 0.196 -> 0.404 | Hurt |
| Add 1 warm-up step (ablation) | DistriFusion on SDXL · 10-step DPM-Solver, COCO Captions 2014 | LPIPS: 0.404 -> 0.288 | Helped |
| Add 2 warm-up steps (ablation) | DistriFusion on SDXL · 10-step DPM-Solver, COCO Captions 2014 | LPIPS: 0.404 -> 0.196 | Helped |
| Use only on-device patch statistics for GroupNorm (ablation) | DistriFusion on SDXL · COCO Captions 2014 val, 50-step DDIM, 8 A100s | LPIPS: 0.211 -> 0.317 | Hurt |
| Reuse stale GroupNorm statistics (ablation) | DistriFusion on SDXL · COCO Captions 2014 val, 50-step DDIM, 8 A100s | LPIPS: 0.211 -> 0.247 | Hurt |
| Synchronize to compute accurate GroupNorm statistics (ablation) | DistriFusion on SDXL · COCO Captions 2014 val, 50-step DDIM, 8 A100s | latency (s): 1.77 -> 1.85 | Hurt |
Surprising findings
- Naïve patch parallelism with 8 devices yields FID 33.6-ish degradation and fragmented outputs, showing patch interaction is essential for quality.
Limitations
- Speedups limited at 1024×1024 by low GPU utilization of SDXL in PyTorch; larger resolutions needed.
- Approximation error grows with fewer sampling steps; needs warm-up steps for ~10-step samplers.
- NCCL communication kernels consume SMs and slow overlapped computation.
Experimental setup
Datasets: COCO Captions 2014
Benchmarks: COCO Captions 2014
Metrics: PSNR LPIPS FID latency MACs speedup
Baselines: Naïve Patch ParaDiGMS Sync. TP Sync. PP Original SDXL
Builds on: SIGE ParaDiGMS Stable Diffusion XL DDIM
Closest papers by meaning
- PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference (NeurIPS 2025 · similarity 0.92)
- Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity (NeurIPS 2024 · similarity 0.91)
- Accelerating Parallel Diffusion Model Serving with Residual Compression (NeurIPS 2025 · similarity 0.91)
- AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising (NeurIPS 2024 · similarity 0.91)
- Hierarchical Patch Diffusion Models for High-Resolution Video Generation (CVPR 2024 · similarity 0.89)
- Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.