Research map / Image generation / Fast diffusion sampling
Optimizing Quantized Diffusion Models via Distillation with Cross-Timestep Error Correction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Cross-Timestep Error Correction (CTEC), a LoRA-based distillation method that trains a quantized diffusion model to also correct errors propagated from the previous timestep, improving low-bit quantized DDIM/LDM image generation.
Problem
Quantized diffusion models accumulate errors across iterative denoising timesteps, and prior quantization methods that only match per-step noise estimates fail to correct this cross-timestep error propagation.
Approach
Derive a corrected distillation target so that the quantized noise predictor also absorbs the previous step's latent error (a term At/Bt (xt - x't)), and learn this target via LoRA-based distillation from the full-precision model rather than conventional scale/zero-point calibration.
Key results
- On LSUN-Bedrooms at W4/A4, CTEC reaches FID 8.52 vs. 10.60 for EfficientDM (LDM-4, 100 DDIM steps).
- On ImageNet at W2/A8 (LDM-4, 20 DDIM steps), CTEC achieves IS 176.37 and FID 7.43, beating EfficientDM (175.03, 7.60).
- Applying the correction term reduces calibration MSE at each timestep by roughly 10x on LSUN at W8/A8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CTEC rather than conventional calibration | Quantized diffusion model · LSUN-Churches, W4/A4 | IS: 2.81 -> 2.89 | Helped |
| Use CTEC rather than conventional calibration | Quantized diffusion model · LSUN-Churches, W4/A4 | FID: 9.08 -> 12.23 | Hurt |
| Use CTEC rather than EfficientDM | Quantized diffusion model · ImageNet, W2/A8 | IS: 175.03 -> 176.37 | Helped |
| Use CTEC rather than EfficientDM | Quantized diffusion model · ImageNet, W2/A8 | FID: 7.60 -> 7.43 | Helped |
| Add CTEC correction to calibration (ablation) | Calibration on LSUN-Churches · LSUN-Churches, W8/A8 | FID: 5.15 -> 8.53 | Hurt |
| Add CTEC correction to distillation without LoRA (ablation) | Distillation on LSUN-Churches · LSUN-Churches, W8/A8 | FID: 5.04 -> 4.62 | Helped |
| Use LoRA for distillation without CTEC (ablation) | Distillation on LSUN-Churches · LSUN-Churches, W8/A8 | FID: 5.04 -> 4.22 | Helped |
| Add CTEC correction to LoRA distillation (ablation) | LoRA distillation on LSUN-Churches · LSUN-Churches, W8/A8 | FID: 4.22 -> 4.00 | Helped |
| Set correction weight γ to 0.6 (ablation) | CTEC · LSUN-Churches and LSUN-Bedrooms, W8/A8 | final-timestep MSE | Helped |
| Apply the CTEC correction term across timesteps (ablation) | Quantized diffusion model · LSUN-Churches and LSUN-Bedrooms, W8/A8 | MSE: approximately 10 times smaller | Helped |
Surprising findings
- Combining the correction term with conventional calibration (scale/zero-point only) degrades performance (FID 8.53 vs 5.15 without correction), showing that distillation is needed to learn the correction.
- At W4/A4 on LSUN-Churches, LSQ achieves a lower FID (9.08) than CTEC (12.23), so CTEC does not win on all FID metrics.
Limitations
- At W4/A4 on LSUN-Churches and Bedrooms, FID remains notably higher than full precision and some baselines (e.g., LSQ).
- Correction weight gamma must be tuned; values above 0.6 cause instability and higher error.
- Evaluation limited to DDIM sampling with LDM models at 256x256 resolution.
Experimental setup
Datasets: LSUN-Churches LSUN-Bedrooms ImageNet
Benchmarks: LSUN-Churches LSUN-Bedrooms ImageNet
Metrics: IS FID sFID Precision MSE calibration error
Baselines: PTQ4DM Q-Diffusion LSQ ADP-DM EfficientDM PTQD
Builds on: DDIM LDM LoRA EfficientDM Q-Diffusion
Closest papers by meaning
- Towards Accurate Post-training Quantization for Diffusion Models (CVPR 2024 · similarity 0.92)
- TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Models (AAAI 2025 · similarity 0.92)
- EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR 2024 · similarity 0.90)
- D^2-DPM: Dual Denoising for Quantized Diffusion Probabilistic Models (AAAI 2025 · similarity 0.90)
- BiDM: Pushing the Limit of Quantization for Diffusion Models (NeurIPS 2024 · similarity 0.90)
- Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compression (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.