Research map / Architectures and efficiency / Quantization
MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MPQ-DM, combining intra-layer kurtosis-driven mixed-precision weight quantization (OMQ) with time-smoothed relation distillation (TRD) to enable extremely low-bit (2-4 bit) quantization of diffusion models.
Problem
Post-training and low-rank fine-tuning quantization of diffusion models degrade severely at extremely low bit-widths (W2-W4), due to weight outlier channels and discretized activations that hinder stable learning across denoising timesteps.
Approach
Allocate bit-widths across channels within each layer (rather than across layers) using kurtosis to identify outlier-salient channels, and distill smoothed multi-step features via KL-based similarity-relation distillation instead of L2 alignment to bridge discrete and continuous latent spaces.
Key results
- Under W2A4 on LSUN-Bedrooms, FID drops by 12.81 versus the EfficientDM baseline, with sFID pushed under 20.
- On ImageNet LDM-4 at W2A4, MPQ-DM reduces FID from 64.45 (EfficientDM) to 36.59, and MPQ-DM+ to 27.11.
- On Stable Diffusion v1.4 at W3A4, CLIP Score rises to 26.96 vs 26.63 for EfficientDM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add outlier-driven mixed quantization (OMQ) (ablation) | LDM-4 baseline · ImageNet 256×256, W3A4 | IS: 134.30 -> 193.18 | Helped |
| Add time-smoothed distillation (TSD) (ablation) | LDM-4 baseline · ImageNet 256×256, W3A4 | IS: 134.30 -> 135.91 | Helped |
| Use OMQ and TSD together (ablation) | LDM-4 baseline · ImageNet 256×256, W3A4 | IS: 134.30 -> 197.43 | Helped |
| Select mixed-bit channels randomly (ablation) | LDM-4 mixed-precision quantization · ImageNet 256×256, W3A4 | FID: 11.08 -> 9.69 | Helped |
| Select channels by kurtosis rather than random or head-tail selection (ablation) | LDM-4 mixed-precision quantization · ImageNet 256×256, W3A4 | FID: 9.69 -> 9.05 | Helped |
| Use N=1 consecutive step for time smoothing instead of N=0 (ablation) | LDM-4 time-smoothed relation distillation · ImageNet 256×256, W3A4 | sFID: 23.37 -> 22.42 | Helped |
| Use N=2, N=3, or N=4 consecutive steps rather than N=0 (ablation) | LDM-4 time-smoothed relation distillation · ImageNet 256×256, W3A4 | sFID: 23.37 -> 23.19; 23.37 -> 22.76; 23.37 -> 22.61 | Helped |
| Replace no distillation with L2 feature alignment (ablation) | LDM-4 distillation · ImageNet 256×256, W3A4 | FID: 9.12 -> 9.13 | Hurt |
| Replace L2 feature alignment with relation distillation (ablation) | LDM-4 distillation · ImageNet 256×256, W3A4 | FID: 9.13 -> 9.10 | Helped |
| Use MPQ-DM rather than the strongest listed baseline | LDM-4 · ImageNet 256×256, W3A4 | FID: 8.49 -> 6.72 | Helped |
| Use MPQ-DM rather than the strongest listed baseline | LDM-4 · ImageNet 256×256, W2A4 | FID: 52.63 -> 36.59 | Helped |
| Use MPQ-DM rather than the strongest listed baseline | LDM-4 · LSUN-Bedrooms 256×256, W3A4 | FID: 19.08 -> 14.80 | Helped |
Surprising findings
- Random or head-tail channel selection for higher bits still gives gains over the baseline, suggesting that raising bits on some channels matters more than lowering others.
Limitations
- MPQ-DM+ needs extra channels at 2-bit, adding a small size overhead and a search-space tradeoff.
- Gains at W2A4 remain large in FID (e.g., 36.59 on ImageNet) relative to FP, so extremely low-bit quality is still far from full precision.
Experimental setup
Datasets: LSUN-Bedrooms LSUN-Churches ImageNet COCO2014
Benchmarks: ImageNet 256x256 LSUN-Bedrooms 256x256 LSUN-Churches 256x256 COCO2014 10k prompts
Metrics: FID sFID IS Precision CLIP Score
Baselines: PTQ-D TFMQ QuEST EfficientDM HAWQ-V3
Builds on: EfficientDM LoRA HAWQ-V3 Q-Diffusion
Closest papers by meaning
- EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR 2024 · similarity 0.92)
- TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Models (AAAI 2025 · similarity 0.91)
- Pioneering 4-Bit FP Quantization for Diffusion Models: Mixup-Sign Quantization and Timestep-Aware Fine-Tuning (CVPR 2025 · similarity 0.91)
- Qua2SeDiMo: Quantifiable Quantization Sensitivity of Diffusion Models (AAAI 2025 · similarity 0.91)
- SVDQuant: Absorbing Outliers by Low-Rank Component for 4-Bit Diffusion Models (ICLR 2025 · similarity 0.90)
- Towards Accurate Post-training Quantization for Diffusion Models (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.