Research map / Architectures and efficiency / Quantization

KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of ViTs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes KroQuant, a learned block-diagonal (32x32) Kronecker-structured invertible activation transform for W4A4 MXFP4 post-training quantization of diffusion transformers, combined with LoRaQ offline weight correction.

Problem

Post-training W4A4 quantization of diffusion transformers suffers from activation outliers; full-dimensional learned transforms are too costly online because AdaLN blocks prevent offline absorption of the transform.

Approach

Parametrize each 32x32 block of the activation transform as a Kronecker product of five 2x2 unit-determinant LU factors (15 parameters per block), initialized at the Hadamard, learned layer-wise with output and regularization losses, and applied as a single tensor-core GEMM per block.

Key results

• On PixArt-Sigma / MJHQ-30K, KroQuant reduces LPIPS from 0.381 (LoRaQ with SmoothQuant) to 0.347 and raises PSNR from 16.4 to 16.5.
• On SDCI with FLUX.1-schnell, KroQuant achieves LPIPS 0.283 vs 0.343 for LoRaQ and PSNR 16.6 vs 15.8.
• On MI350, the KroQuant K1 kernel is 1.00-1.14x as fast as per-channel scaling across tested shapes, and 1.30-2.45x faster than FWHT for K2.

Surprising findings

• A fixed 32x32 Hadamard transform with LoRaQ matches SmoothQuant on FID but degrades PSNR (15.7 vs 16.4).
• KroQuant standalone (without LoRaQ) gives FID 41.8 vs 130.6 for smoothing, but mixed results on FLUX.1 MJHQ-30K where it fails to beat SVDQuant on IR and LoRaQ on LPIPS.

Limitations

• Kronecker parametrization has only 3 log2(n) parameters per block, limiting expressivity at larger block sizes; tied to n=32 in practice.
• Joint multi-layer optimization is unstable, partly due to the unit-determinant LU parametrization.
• On FLUX.1-schnell MJHQ-30K the method does not improve over SVDQuant on IR or LoRaQ on LPIPS.

Experimental setup

Datasets: COCO 2017 validation captions

Benchmarks: MJHQ-30K SDCI

Metrics: FID Image Reward LPIPS PSNR kernel latency

Baselines: SVDQuant LoRaQ SmoothQuant Hadamard (32x32) block transform FWHT

Builds on: SmoothQuant LoRaQ SVDQuant Hadamard transform

Closest papers by meaning

• DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers (NeurIPS 2026 · similarity 0.91)
• PTQ4DiT: Post-training Quantization for Diffusion Transformers (NeurIPS 2024 · similarity 0.91)
• Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers (CVPR 2025 · similarity 0.91)
• Qua2SeDiMo: Quantifiable Quantization Sensitivity of Diffusion Models (AAAI 2025 · similarity 0.89)
• VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers (AAAI 2025 · similarity 0.89)
• EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR 2024 · similarity 0.89)