Research map / Image generation / Fast diffusion sampling
Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Q-DiT proposes a post-training quantization framework for diffusion transformers that combines evolutionary search for per-layer group sizes with sample-wise dynamic activation quantization.
Problem
Post-training quantization of diffusion transformers (DiTs) suffers large degradation because DiTs show input-channel variance in weights and activations and activation distribution shifts across timesteps and samples, which UNet-oriented PTQ methods do not handle.
Approach
Fine-grained group quantization with per-layer group sizes chosen by an evolutionary search that optimizes FID/FVD under a BitOps constraint, plus on-the-fly min-max activation quantization computed per sample and timestep rather than calibrated static parameters.
Key results
- On ImageNet 256x256 at W6A8, Q-DiT reaches FID 12.21 vs 12.40 for full precision DiT-XL/2 (100 steps, no cfg).
- At W4A8 on ImageNet 256x256 (100 steps, cfg 1.5), Q-DiT achieves FID 6.40 vs 5.31 full precision, versus 7.66 for G4W+P4A and 7.75 for PTQ4DiT.
- On VBench video generation at W4A8 with STDiT3, Q-DiT outperforms G4W+P4A on 15 of 16 dimensions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Q-DiT W6A8 instead of G4W+P4A W6A8 | DiT-XL/2 · ImageNet 256×256, 100 steps | FID: 16.72 -> 12.21 | Helped |
| Use Q-DiT W4A8 instead of G4W+P4A W4A8 | DiT-XL/2 · ImageNet 256×256, 100 steps | FID: 25.48 -> 15.76 | Helped |
| Use Q-DiT W4A8 instead of G4W+P4A W4A8 | DiT-XL/2 · ImageNet 512×512, 50 steps, cfg 1.5 | FID: 9.98 -> 7.82 | Helped |
| Use Q-DiT W4A8 instead of G4W+P4A W4A8 | STDiT3 · VBench video generation | Subject Consistency: 0.9444 -> 0.9498 | Helped |
| Add group size 128 to W4A8 RTN (ablation) | DiT-XL/2 · ImageNet 256×256, 100 steps, cfg 1.5 | FID: 225.50 -> 13.77 | Helped |
| Add sample-wise dynamic activation quantization to group size 128 (ablation) | DiT-XL/2 · ImageNet 256×256, 100 steps, cfg 1.5 | FID: 13.77 -> 6.64 | Helped |
| Add automatic quantization granularity allocation (ablation) | DiT-XL/2 · ImageNet 256×256, 100 steps, cfg 1.5 | FID: 6.64 -> 6.40 | Helped |
| Use sample-wise and timestep-wise dynamic activation quantization instead of TFMQ-DM | DiT-XL/2 W16A8 · ImageNet 256×256, 100 steps, cfg 1.5 | FID: 7.74 -> 5.34 | Helped |
| Use evolutionary group-size search instead of fixed group size 128 (ablation) | DiT-XL/2 · ImageNet 256×256, 100 steps, cfg 1.5 | FID: 6.64 -> 6.40 | Helped |
| Use evolutionary group-size search instead of ILP search (ablation) | DiT-XL/2 · ImageNet 256×256, 100 steps, cfg 1.5 | FID: 6.71 -> 6.40 | Helped |
| Use evolutionary group-size search instead of Hessian-based search (ablation) | DiT-XL/2 · ImageNet 256×256, 100 steps, cfg 1.5 | FID: 7.38 -> 6.40 | Helped |
Surprising findings
- Smaller group sizes do not always improve FID (e.g., group 128 gives FID 17.87 while group 96 gives 19.97), showing non-monotonicity.
- Lower MSE between quantized and full-precision layers does not reliably correspond to lower generation degradation for DiTs, so MSE-based ILP sensitivity methods are ineffective.
Limitations
- Evolutionary group-size search is computationally expensive and time-consuming.
- Evaluated only on DiT-XL/2 for images and STDiT3 for video.
- Search uses small sample sizes (500 images for FID, 100 videos for FVD) for proxy evaluation.
Experimental setup
Datasets: ImageNet VBench prompt suite
Benchmarks: ImageNet 256x256 ImageNet 512x512 VBench
Metrics: FID sFID IS Precision FVD VBench dimensions
Baselines: PTQ4DM RepQ-ViT TFMQ-DM PTQ4DiT G4W+P4A RTN ILP Hessian-based search
Builds on: GPTQ PTQ4DM RepQ-ViT TFMQ-DM PTQ4DiT
Closest papers by meaning
- PTQ4DiT: Post-training Quantization for Diffusion Transformers (NeurIPS 2024 · similarity 0.95)
- ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation (ICLR 2025 · similarity 0.93)
- TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Models (AAAI 2025 · similarity 0.93)
- DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers (NeurIPS 2026 · similarity 0.93)
- EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR 2024 · similarity 0.92)
- VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers (AAAI 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.