Research map / Video / Video generation
Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Q-VDiT, a post-training quantization framework for video diffusion transformers that combines a Token-aware Quantization Estimator (low-rank weight/activation error compensation) with Temporal Maintenance Distillation (KL alignment of inter-frame similarity distributions).
Problem
Quantizing video-generation diffusion transformers to low bit-widths causes large quality loss, because existing image-DiT quantization methods ignore the higher information density of video tokens and use per-output MSE objectives that ignore inter-frame correlations.
Approach
Estimate quantization error with a rank-1 token- and feature-aware low-rank correction (TQE), and add a KL loss that aligns the frame-to-frame similarity distributions of the quantized and FP models so each frame is optimized with respect to the whole video (TMD).
Key results
- Under W3A6 on VBench, Q-VDiT raises scene consistency to 23.40 versus 12.04 for the best prior method (about 1.9x).
- Under W3A6 on the OpenSORA prompt set, VQA-Technical improves from 29.58 (best prior, EfficientDM) to 59.10.
- At W4A8 on Open-Sora, Q-VDiT reaches VQA-Aesthetic 71.32, above the FP16 model's 66.91.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Q-VDiT instead of EfficientDM | Open-Sora W3A6 · VBench | Scene Consist.: 12.04 to 23.40 | Helped |
| use Q-VDiT instead of EfficientDM | Open-Sora W3A6 · OpenSORA prompt set | VQA-Technical: 29.58 to 59.10 | Helped |
| use Q-VDiT instead of SmoothQuant | Open-Sora W3A6 · OpenSORA prompt set | CLIPSIM: 0.1768 to 0.1785 | Helped |
| use Q-VDiT instead of ViDiT-Q | Latte · UCF-101 W3A6 | VQA-Technical: 19.61 to 25.81 | Helped |
| add TQE without token scaling M (ablation) | PTQ4DiT · W3A6 Open-Sora | Scene Consist.: 7.85 to 21.94 | Helped |
| add token scaling M to TQE (ablation) | TQE without M · W3A6 Open-Sora | Scene Consist.: 21.94 to 22.38 | Helped |
| add TMD (ablation) | TQE with token scaling M · W3A6 Open-Sora | Scene Consist.: 22.38 to 22.87 | Helped |
| combine TQE and TMD (ablation) | Q-VDiT · W3A6 Open-Sora | Scene Consist.: 22.87 to 23.40 | Helped |
Surprising findings
- At W4A8, Q-VDiT's VQA metrics exceed the full-precision model's scores on Open-Sora.
- Under W8A8, ViDiT-Q (no calibration) achieves VQA-Aesthetic 60.70, close to other calibrated methods, and the paper's gains are mainly in quality rather than efficiency.
Limitations
- Calibration uses only 10 prompts from the Open-Sora prompt set.
- Evaluation is limited to two video DiTs (Open-Sora and Latte) and mostly automatic metrics without human evaluation.
- FVD on the 101-video UCF-101 setting is noted by the authors as unstable, motivating the proposed FVD-FP16 metric.
Experimental setup
Datasets: Open-Sora prompt set (10 prompts for calibration) VBench prompt suites UCF-101
Benchmarks: VBench OpenSORA prompt set multi-aspect metrics UCF-101 (Latte)
Metrics: Imaging Quality Aesthetic Quality Motion Smoothness Dynamic Degree Background Consistency Subject Consistency Scene Consistency Overall Consistency CLIPSIM CLIP-Temp
Baselines: Q-DiT PTQ4DiT SmoothQuant Quarot EfficientDM SVDQuant ViDiT-Q Q-Diffusion Naive-PTQ
Builds on: ViDiT-Q PTQ4DiT EfficientDM SVDQuant LoRA
Closest papers by meaning
- ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation (ICLR 2025 · similarity 0.95)
- $\text{S}^2$Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation (NeurIPS 2025 · similarity 0.92)
- DSAQuant: Denoising-Stage-Aligned Quantization-Aware Training for Video Generation (NeurIPS 2026 · similarity 0.92)
- VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers (AAAI 2025 · similarity 0.92)
- Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers (CVPR 2025 · similarity 0.91)
- Towards Precise Scaling Laws for Video Diffusion Transformers (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.