Research map / Image generation / Fast diffusion sampling
AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AccuQuant, a post-training quantization method for diffusion models that calibrates quantization step-sizes by matching full-precision and quantized outputs over groups of several denoising steps, with an approximated gradient giving O(1) memory.
Problem
Quantization errors accumulate across denoising steps in diffusion sampling, but existing PTQ methods calibrate each step independently and ignore this accumulated error.
Approach
Group consecutive denoising steps and minimize the discrepancy between full-precision and quantized outputs over the group, using separate inputs for each model; a gradient approximation (dropping the noise-Jacobian term) and a stop-gradient surrogate loss reduce memory from O(n) to O(1) in group size.
Key results
- On CIFAR-10 DDIM at 4/8-bit (W/A), FID2FP32 of 1.15 versus 3.26 for Q-Diffusion and 1.52 for TFMQ-DM.
- On LSUN-Churches LDM-8 at 4/8-bit, FID2FP32 of 0.66 versus 0.79 for Q-Diffusion; on ImageNet LDM-4 at 3/8-bit, FID2FP32 of 3.86 versus 11.26 for Q-Diffusion.
- Gradient approximation gives better FID and FID2FP32 than the exact gradient on CIFAR-10 at group size 5 while reducing memory to be constant in group size.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Simulate multiple denoising steps during calibration instead of calibrating steps independently | AccuQuant · CIFAR-10 DDIM; LSUN-Churches; LSUN-Bedrooms; ImageNet; MS-COCO | FID2FP32: 35.24 -> 3.30; 0.79 -> 0.66; 1.92 -> 1.07; 1.71 -> 0.65; 20.42 -> 10.99 | Helped |
| Add gradient approximation (ablation) | AccuQuant, DDIM on CIFAR-10 · CIFAR-10, group size 5, 6/6 | FID: 6.13 -> 5.79 | Helped |
| Add gradient approximation (ablation) | AccuQuant, DDIM on CIFAR-10 · CIFAR-10, group size 5, 4/8 | FID2FP32: 2.32 -> 1.15 | Helped |
| Add gradient approximation (ablation) | AccuQuant, DDIM on CIFAR-10 · CIFAR-10, group size 5, 4/6 | FID2FP32: 5.18 -> 3.94 | Helped |
| Add gradient approximation (ablation) | AccuQuant, DDIM on CIFAR-10 · CIFAR-10, group size 5, 3/8 | FID2FP32: 5.34 -> 5.15 | Helped |
| Add gradient approximation (ablation) | AccuQuant, DDIM on CIFAR-10 · CIFAR-10, group size 5, 3/6 | FID2FP32: 7.80 -> 6.69 | Helped |
| Use group size 1 instead of group size 5 (ablation) | AccuQuant, DDIM on CIFAR-10, 4/8 · CIFAR-10 | FID2FP32: 1.15 -> 3.11 | Hurt |
| Use group size 10 instead of group size 5 (ablation) | AccuQuant, DDIM on CIFAR-10, 4/8, with gradient approximation · CIFAR-10 | FID: 4.75 -> 5.07 | Hurt |
| Use group size 25 instead of group size 5 (ablation) | AccuQuant, DDIM on CIFAR-10, 4/8, with gradient approximation · CIFAR-10 | FID: 4.75 -> 5.24 | Hurt |
| Use group size 25 instead of group size 5 (ablation) | AccuQuant, DDIM on CIFAR-10, 4/8, with gradient approximation · CIFAR-10 | FID2FP32: 1.15 -> 1.49 | Mixed |
| Use group size 100 instead of group size 5 (ablation) | AccuQuant, DDIM on CIFAR-10, 6/6 · CIFAR-10 | image fidelity | Hurt |
| Use group size 1 instead of group size 5 (ablation) | AccuQuant, DDIM on CIFAR-10, 4/8, without gradient approximation · CIFAR-10 | FID2FP32: 2.32 -> 3.11 | Hurt |
Surprising findings
- Dropping the noise-Jacobian term of the gradient improves calibration stability and final results rather than hurting them.
Limitations
- Requires multiple denoising steps during calibration, so it may be less effective for 1-2 step diffusion models.
- Group size is a fixed hyperparameter that must be tuned across models and datasets.
- Calibration cost remains similar to prior methods, so gains are in accuracy rather than speed.
Experimental setup
Datasets: CIFAR-10 LSUN-Bedrooms LSUN-Churches ImageNet MS-COCO
Benchmarks: CIFAR-10 LSUN-Bedrooms LSUN-Churches ImageNet MS-COCO
Metrics: FID IS sFID FID2FP32 LPIPS PSNR SSIM CLIP score
Baselines: Q-Diffusion TFMQ-DM TAC PTQD PCR PTQ4DM EfficientDM
Builds on: Q-Diffusion DDIM AdaRound TFMQ-DM PCR
Closest papers by meaning
- D^2-DPM: Dual Denoising for Quantized Diffusion Probabilistic Models (AAAI 2025 · similarity 0.91)
- Qua2SeDiMo: Quantifiable Quantization Sensitivity of Diffusion Models (AAAI 2025 · similarity 0.91)
- StepbaQ: Stepping backward as Correction for Quantized Diffusion Models (NeurIPS 2024 · similarity 0.91)
- Towards Accurate Post-training Quantization for Diffusion Models (CVPR 2024 · similarity 0.91)
- EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR 2024 · similarity 0.91)
- MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.