Research map / Architectures and efficiency / Quantization

DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of ViTs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DiRotQ, a W4A4 post-training quantization method for diffusion transformers that rotates activations into a per-layer PCA basis, keeps the top ~10% components in high precision, and quantizes the residual to 4-bit with an extra random rotation and GPTQ weights, plus a VLM-as-a-judge evaluation and a Triton kernel.

Problem

Post-training 4-bit (W4A4) quantization of diffusion transformers causes large quality degradation relative to FP16/BF16, and existing rotation-based LLM methods were thought inapplicable to DiTs due to AdaLN and timestep-dependent activation statistics.

Approach

Compute a per-layer PCA basis from calibration activations averaged across timesteps, keep high-variance PCA components in high precision, and quantize the low-variance residual after an additional random orthogonal rotation; the inverse rotation is fused into weights offline so only the forward rotation runs online.

Key results

• On PixArt-Σ INT W4A4 (MJHQ-30K), DiRotQ achieves FID 15.9 and PSNR 19.1 vs. SVDQuant's FID 18.9 and PSNR 17.6.
• On FLUX.1-dev, DiRotQ reduces memory by 2.1x and gives 2.3x per-step speedup over BF16 at batch size 1 on an RTX 4090.
• Ablation on PixArt-Σ INT W4A4: RTN gives FID 40.3, PCA rotation alone 19.7, adding residual rotation 19.2, and adding GPTQ 15.9.

Surprising findings

• Naive activation-space PCA with 10% FP16 tail already surpasses RTN by 10-20 dB QSNR, with the rotation rather than the tail size driving most gains.
• Reported 9.2x speedup at batch size 2 is largely attributed to eliminating CPU offloading of BF16 model rather than kernel speed alone.

Limitations

• Single shared PCA basis across timesteps; per-timestep rotations would increase weight memory and are not explored.
• Evaluated only on text-to-image; video not addressed.
• VLM judges are open-source substitutes for GPT-4o, with weaker correlation to human judgment than the original VIEScore setup.

Experimental setup

Datasets: COCO Captions MJHQ-30K sDCI

Benchmarks: MJHQ-30K sDCI

Metrics: FID Image Reward LPIPS PSNR CLIP IQA CLIP Score SSIM VLM-as-a-judge SC/PQ/Overall

Baselines: PTQ4DiT Q-DiT ViDiT-Q SVDQuant RTN

Builds on: SVDQuant ViDiT-Q QuaRot ResQ GPTQ VIEScore

Closest papers by meaning

• PTQ4DiT: Post-training Quantization for Diffusion Transformers (NeurIPS 2024 · similarity 0.93)
• Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers (CVPR 2025 · similarity 0.93)
• VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers (AAAI 2025 · similarity 0.92)
• KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers (NeurIPS 2026 · similarity 0.91)
• ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation (ICLR 2025 · similarity 0.91)
• EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR 2024 · similarity 0.90)