Research map / Architectures and efficiency / Quantization
Qua2SeDiMo: Quantifiable Quantization Sensitivity of Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Qua²SeDiMo, a mixed-precision post-training weight quantization framework for diffusion denoisers that uses a GNN explanation method over denoiser graphs to attribute end-to-end FID to individual layers and blocks, yielding sub-4-bit configurations for PixArt, Hunyuan-DiT and SDXL.
Problem
Quantization sensitivity of individual weight layers, operation types and block structures in diffusion denoisers (U-Nets and DiTs) is poorly understood, and existing PTQ methods struggle below 4-bit weight precision without degrading image quality.
Approach
Cast denoisers as DAGs with per-layer quantization settings as node features, train a GNN to predict FID (plus bit cost) with an added ranking loss on hop-level embedding norms, and use those norms as sensitivity scores to pick per-layer or per-block bit-precision and quantization method without a calibration set.
Key results
- On PixArt-α, the W4A16 Qua²SeDiMo configuration reaches FID 29.02 versus 33.98 for ViDiT-Q and 41.93 for Q-Diffusion; the W3.4A16 configuration reaches FID 35.51.
- On PixArt-Σ, a W3.9A16 configuration achieves FID 30.34 versus 31.21 for ViDiT-Q (second best) at the same activation precision.
- In a 20-person user study on PixArt-α W4A8 images, Qua²SeDiMo received 43.22% of votes versus 28.81% for Q-Diffusion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Qua²SeDiMo W4A16 instead of full precision | PixArt-α · COCO 2014 | FID: 34.05 -> 29.02 | Helped |
| Use Qua²SeDiMo W4A16 instead of ViDiT-Q W4A16 | PixArt-α · COCO 2014 | FID: 33.98 -> 29.02 | Helped |
| Reduce Qua²SeDiMo weight precision from W4 to W3.7 | PixArt-α · COCO 2014, A16 | FID: 29.02 -> 33.06 | Hurt |
| Use Qua²SeDiMo W4A8 instead of ViDiT-Q W4A8 | PixArt-α · COCO 2014 | FID: 39.11 -> 38.39 | Helped |
| Reduce activation precision to A6 in Qua²SeDiMo W4 | PixArt-α · COCO 2014 | FID: 38.39 -> 54.59 | Hurt |
| Use Qua²SeDiMo W3.9A16 instead of ViDiT-Q W4A16 | PixArt-Σ · COCO 2014 | FID: 31.21 -> 30.34 | Helped |
| Use Qua²SeDiMo W3.9A8 instead of Q-Diffusion OAQ W4A8 | PixArt-Σ · COCO 2014 | FID: 36.89 -> 35.61 | Helped |
| Use Qua²SeDiMo W4A16 instead of Q-Diffusion W4A16 | Hunyuan-DiT · COCO 2014 | FID: 42.09 -> 40.25 | Helped |
| Use Qua²SeDiMo W4A8 instead of TFMQ-DM W4A8 | Hunyuan-DiT · COCO 2014 | FID: 72.15 -> 51.91 | Helped |
| Use Qua²SeDiMo W4A16 instead of full precision | SDv1.5 · COCO 2014 | FID: 18.32 -> 18.28 | Helped |
| Use block-level optimization with NDCG or Hybrid loss rather than SRCC loss | Qua²SeDiMo · PixArt-α, PixArt-Σ, Hunyuan, and SDXL constrained FID/average-bits optimization | FID and average bits | Helped |
| Use operation-level optimization rather than block-level optimization | Qua²SeDiMo · PixArt-α, PixArt-Σ, Hunyuan, and SDXL constrained FID/average-bits optimization | FID and average bits | Hurt |
Surprising findings
- Operation-level optimization often fails outright or fixates on high-bit configurations, while block-level subgraph optimization with NDCG or Hybrid losses performs consistently better.
- Q-Diffusion's original activation quantization catastrophically fails (FID 373.62 on PixArt-α W4A8) for DiTs, requiring online ZeroQuant activation quantization to be competitive.
Limitations
- Search space is restricted to 3- and 4-bit weights with three quantization methods, so the configurations are not evaluated beyond these choices.
- Predictors achieve only moderate validation performance (around 0.5 SRCC or NDCG@10), and block subgraph design requires extensive manual choices.
- FID is sensitive to prompt set and base model, so cross-model FID comparisons are limited.
Experimental setup
Datasets: MS-COCO 2017 validation MS-COCO 2014 validation ImageNet validation
Benchmarks: MS-COCO 2014 (10k images) MS-COCO 2017 validation ImageNet
Metrics: FID CLIP score average bit precision user preference votes
Baselines: Q-Diffusion TFMQ-DM ViDiT-Q K-Means UAQ
Builds on: Q-Diffusion ZeroQuant LambdaRank GATv2 AdaRound
Closest papers by meaning
- EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR 2024 · similarity 0.92)
- Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers (CVPR 2025 · similarity 0.92)
- AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models (NeurIPS 2025 · similarity 0.91)
- D^2-DPM: Dual Denoising for Quantized Diffusion Probabilistic Models (AAAI 2025 · similarity 0.91)
- MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models (AAAI 2025 · similarity 0.91)
- Pioneering 4-Bit FP Quantization for Diffusion Models: Mixup-Sign Quantization and Timestep-Aware Fine-Tuning (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.