Research map / Image generation / Fast diffusion sampling
TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TFMQ-DM, a post-training quantization framework for diffusion models that isolates a time-step-only Temporal Information Block and applies temporal information aware reconstruction (TIAR) for weights and finite set calibration (FSC) for activations, achieving near-FP quality at W4A8.
Problem
Post-training quantization of diffusion models degrades quality, especially at low bit-width, because quantization disturbs the time-step-dependent temporal features that control the denoising trajectory.
Approach
Identify the embedding layers and time-embedding modules that depend only on the time-step as a separate Temporal Information Block; reconstruct it with a temporal-feature loss (TIAR) and calibrate its activations with a separate set of quantization parameters per time-step (FSC), since the set of time-steps is finite.
Key results
- On LSUN-Bedrooms 256x256 with LDM-4 at W4A8, TFMQ-DM reaches FID 3.68 vs 2.98 full precision and 9.36 for BRECQ baseline (ablation), with sFID 7.65.
- On CelebA-HQ 256x256 at W4A8, FID is reduced by 6.71 relative to the prior SOTA.
- Quantization time is reduced by 2.0x on LSUN-Bedrooms (2.32 GPU hours vs 4.68 for PTQD and 5.29 for Q-Diffusion).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TFMQ-DM instead of PTQD at 4/8-bit quantization | LDM-4 · LSUN-Bedrooms 256 × 256 | FID: 5.94 -> 3.68 | Helped |
| Use TFMQ-DM instead of PTQD at 4/8-bit quantization | LDM-4 · LSUN-Bedrooms 256 × 256 | sFID: 15.16 -> 7.65 | Helped |
| Use TFMQ-DM instead of PTQD at 4/8-bit quantization | LDM-4 · CelebA-HQ 256 × 256 | FID: 15.47* -> 8.76 | Helped |
| Use TFMQ-DM instead of PTQD at 4/8-bit quantization | LDM-8 · ImageNet 256 × 256 | FID: 10.40 -> 10.29 | Helped |
| Use TFMQ-DM instead of PTQD at 4/8-bit quantization | LDM-8 · ImageNet 256 × 256 | sFID: 12.63 -> 7.35 | Helped |
| Use TFMQ-DM instead of PTQD at 4/8-bit quantization | LDM-8 · ImageNet 256 × 256 | IS: 214.73 -> 221.82 | Helped |
| Use TFMQ-DM instead of Q-Diffusion at 4/8-bit quantization | Stable-Diffusion-v1-4 · MS-COCO | FID: 14.49 -> 13.36 | Helped |
| Use TFMQ-DM instead of Q-Diffusion at 4/8-bit quantization | Stable-Diffusion-v1-4 · MS-COCO | CLIP: 0.3121 -> 0.3128 | Helped |
| Add temporal information aware reconstruction (TIAR) (ablation) | BRECQ on LDM-4 · LSUN-Bedrooms 256 × 256 | FID: 9.36 -> 4.84 | Helped |
| Add temporal information aware reconstruction (TIAR) (ablation) | BRECQ on LDM-4 · LSUN-Bedrooms 256 × 256 | sFID: 22.73 -> 9.29 | Helped |
| Add finite set calibration (FSC) (ablation) | BRECQ on LDM-4 · LSUN-Bedrooms 256 × 256 | FID: 9.36 -> 6.07 | Helped |
| Add finite set calibration (FSC) (ablation) | BRECQ on LDM-4 · LSUN-Bedrooms 256 × 256 | sFID: 22.73 -> 11.31 | Helped |
Surprising findings
- LSQ, the only learning-based calibration tested, did not outperform simple min-max calibration and sometimes performed worse.
- Quantized TFMQ-DM at W4A32 on ImageNet and W8A8 on Stable Diffusion FID is even lower than the full-precision model.
Limitations
- Focused only on PTQ; extension to QAT left to future work.
- Textual features in Stable Diffusion are not considered by the method.
- Gains on LDM-8 LSUN-Churches are moderate, as existing methods already perform well there.
Experimental setup
Datasets: CIFAR-10 LSUN-Bedrooms 256x256 LSUN-Churches 256x256 CelebA-HQ 256x256 ImageNet 256x256 FFHQ 256x256 MS-COCO
Benchmarks: CIFAR-10 LSUN-Bedrooms 256x256 LSUN-Churches 256x256 CelebA-HQ 256x256 FFHQ 256x256 ImageNet 256x256 MS-COCO
Metrics: FID sFID IS CLIP score GPU hours
Baselines: PTQ4DM Q-Diffusion PTQD BRECQ ADP-DM TDQ
Builds on: BRECQ AdaRound Q-Diffusion PTQD LDM
Closest papers by meaning
- TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Models (AAAI 2025 · similarity 0.91)
- MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models (AAAI 2025 · similarity 0.90)
- Towards Accurate Post-training Quantization for Diffusion Models (CVPR 2024 · similarity 0.90)
- EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR 2024 · similarity 0.90)
- AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models (NeurIPS 2025 · similarity 0.89)
- Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.