Research map / Architectures and efficiency / Quantization

SVDQuant: Absorbing Outliers by Low-Rank Component for 4-Bit Diffusion Models

ICLR 2025 · Architectures and efficiency › Quantization · approach: Post-training quantization of ViTs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SVDQuant, a W4A4 post-training quantization paradigm for diffusion models that migrates activation outliers into weights via smoothing and absorbs them with a 16-bit SVD low-rank branch, plus the Nunchaku kernel-fusion engine to make the branch cheap.

Problem

Post-training 4-bit quantization of both weights and activations in diffusion models causes severe quality loss, and existing smoothing methods are insufficient; naively adding a low-rank branch also adds substantial latency overhead.

Approach

Smooth activation outliers into weights, then decompose the smoothed weight with SVD into a 16-bit low-rank branch (absorbing weight outliers) and a residual quantized to 4 bits, and fuse the low-rank kernels into the quantize and 4-bit compute kernels to remove redundant memory access.

Key results

• On FLUX.1-dev MJHQ, NVFP4 W4A4 with SVDQuant reaches FID 20.4 and ImageReward 0.937 versus NF4 W4A16 at 20.6 and 0.910.
• On a laptop RTX 4090, the INT4 FLUX.1 model runs 3.0x faster than NF4 W4A16 and reduces transformer size from 22.2 GiB to 6.1 GiB (3.6x reduction).
• On PixArt-Sigma, the naive W4A4 INT4 baseline with ViDiT-Q degrades to FID 412, while SVDQuant INT4 reaches FID 19.2 and NVFP4 reaches FID 16.6 (FP16: 16.6).

Measured effects reported in the paper

Surprising findings

• Naive W4A4 quantization of PixArt-Sigma with ViDiT-Q collapses (FID 412), while the same setting with SVDQuant recovers near-FP16 quality.
• Running the low-rank branch naively adds about 57% latency overhead, which the kernel fusion in Nunchaku reduces to 5-10%.

Limitations

• Below 3 bits (e.g., W2A4, W4A2) the method cannot generate usable images.
• Quality for the smaller PixArt-Sigma model with INT4 shows slight degradation relative to FP16 and relies on NVFP4 for near-parity.
• Adding the low-rank branch still adds 5-10% latency and rank increases trade off quality against overhead.

Experimental setup

Datasets: COCO Captions 2024 MJHQ-30K sDCI

Benchmarks: MJHQ sDCI

Metrics: FID ImageReward LPIPS PSNR CLIP IQA CLIP Score SSIM latency memory

Baselines: NF4 W4A16 ViDiT-Q MixDQ TensorRT SmoothQuant LoRC

Builds on: SmoothQuant GPTQ LoRC QServe

Closest papers by meaning

• EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR 2024 · similarity 0.91)
• MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models (AAAI 2025 · similarity 0.90)
• Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models (ICML 2025 · similarity 0.90)
• Qua2SeDiMo: Quantifiable Quantization Sensitivity of Diffusion Models (AAAI 2025 · similarity 0.90)
• AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models (NeurIPS 2025 · similarity 0.89)
• VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers (AAAI 2025 · similarity 0.89)