Research map / Architectures and efficiency / Quantization

$\text{S}^2$Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation

NeurIPS 2025 · Architectures and efficiency › Quantization · approach: Post-training quantization of ViTs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes S²Q-VDiT, a post-training quantization framework for video diffusion transformers that selects salient calibration data via Hessian-aware scoring and reweights token-wise quantization loss by attention, achieving near-lossless W4A6 quantization of CogVideoX-2B/5B and HunyuanVideo.

Problem

Post-training quantization of video diffusion transformers is hard because long spatio-temporal token sequences limit the calibration set to a few dozen samples, making quantization quality highly sensitive to calibration data and treating all tokens equally in the loss is suboptimal.

Approach

Jointly scores calibration samples by diffusion salience (normalized change between consecutive timesteps) and quantization salience (Hessian-based sensitivity) via a product metric, and reweights per-token quantization loss by attention received from other tokens so that influential tokens dominate block-wise calibration.

Key results

• Under W4A6, S²Q-VDiT on CogVideoX-5B reaches Imaging Quality 60.75 vs FP 61.80 and best baseline ~48.87 (ViDiT-Q), described as near-lossless.
• Under W4A4 on CogVideoX-2B, Imaging Quality 53.71 vs 45.56 for ViDiT-Q and 26.26 for Q-DiT.
• On CogVideoX-5B W4A6, achieves 3.94x model storage compression and 1.28x inference acceleration (latency 259.2s to 203.2s at batch size 1) on a single A800.

Measured effects reported in the paper

Surprising findings

• Some baselines achieve high Subject and Background Consistency under W4A4 only because their generations have nearly collapsed, so these metrics are misleading for collapsed models.
• Under W4A6 on CogVideoX-5B, S²Q-VDiT's Scene Consistency (46.66) exceeds the FP model's 45.28.

Limitations

• Not fully lossless under the most difficult W4A4 setting.
• Calibration set limited to about 40 samples due to long token sequences; performance gains saturate beyond that.
• Evaluation relies on VBench and EvalCrafter with a small number of prompts and sample videos per setting.

Experimental setup

Datasets: VBench prompt sets EvalCrafter prompt sets ViDiT-Q prompt sets

Benchmarks: VBench EvalCrafter

Metrics: Imaging Quality Aesthetic Quality Motion Smoothness Dynamic Degree Background Consistency Subject Consistency Scene Consistency Overall Consistency CLIPSIM CLIP-Temp

Baselines: Q-DiT PTQ4DiT ViDiT-Q SmoothQuant QuaRot

Builds on: ViDiT-Q PTQ4DiT QuaRot GPTQ SmoothQuant

Closest papers by meaning

• Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformers (ICML 2025 · similarity 0.92)
• ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation (ICLR 2025 · similarity 0.92)
• Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers (CVPR 2025 · similarity 0.92)
• VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers (AAAI 2025 · similarity 0.91)
• DSAQuant: Denoising-Stage-Aligned Quantization-Aware Training for Video Generation (NeurIPS 2026 · similarity 0.91)
• PTQ4DiT: Post-training Quantization for Diffusion Transformers (NeurIPS 2024 · similarity 0.90)