Research map / Architectures and efficiency / Quantization

Outlier-Aware Post-Training Quantization for Discrete Graph Diffusion Models

ICML 2025 · Architectures and efficiency › Quantization · approach: Post-training quantization of ViTs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Bit-DGDM, a post-training quantization framework for discrete graph diffusion models that splits activations into sparse outliers plus low-bit dense parts and decomposes weights into low-rank plus alpha-sparse outlier components, with custom sparse-dense kernels.

Problem

Discrete graph diffusion models are compute-bound with outliers in both weights and activations, and existing PTQ methods designed for LLMs and image diffusion models do not transfer well to them.

Approach

Combine data-free adaptive outlier thresholds for activations (sparse-dense activation quantization) with an ill-conditioned, RPCA-style low-rank plus alpha-sparse weight decomposition, and implement equidistributed sparse-dense GPU kernels so that the dense low-bit part dominates computation.

Key results

• On QM9 Digress, Bit-DGDM W4A4 reaches 98.2% validity and 2.5x speedup versus 99.0% and 1x for FP32, with 2.3 GB memory vs 6.4 GB.
• On CATH inverse folding, Bit-DGDM W4A4 gets perplexity 4.5 and recovery 51.6% vs FP32 4.4 and 52.2%, with 2.5x speedup.
• Outperforms DuQuant (2.1x speedup) and SVDQuant on QM9 and MOSES at W4A4.

Measured effects reported in the paper

Surprising findings

• Weight-only SqueezeLLM gives little speedup (about 1.0-1.1x), suggesting weight loading is a minor bottleneck for DGDMs compared with computation.

Limitations

• Relies on theoretical recovery guarantees whose assumptions (mu-incoherence, alpha-sparsity bound) are not verified in practice for the models.
• Evaluated only on Digress and GRADE-IF; generalization to other DGDMs is claimed but limited.

Experimental setup

Datasets: QM9 MOSES SBM planar graphs CATH

Benchmarks: QM9 MOSES SBM planar CATH

Metrics: Validity Uniqueness Novelty Speedup Peak memory Perplexity Recovery

Baselines: RTN GPTQ SqueezeLLM SVDQuant DuQuant BF16

Builds on: Digress GRADE-IF robust principal component analysis SVDQuant GPTQ

Closest papers by meaning

• SVDQuant: Absorbing Outliers by Low-Rank Component for 4-Bit Diffusion Models (ICLR 2025 · similarity 0.90)
• MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models (AAAI 2025 · similarity 0.90)
• DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Models (ICLR 2025 · similarity 0.90)
• EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR 2024 · similarity 0.90)
• Qua2SeDiMo: Quantifiable Quantization Sensitivity of Diffusion Models (AAAI 2025 · similarity 0.89)
• Towards Accurate Post-training Quantization for Diffusion Models (CVPR 2024 · similarity 0.88)