Research map / Image generation / Fast diffusion sampling
BiDM: Pushing the Limit of Quantization for Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BiDM, the first method to fully binarize (W1A1) diffusion models, using a Timestep-friendly Binary Structure (learnable activation scaling plus cross-timestep connections) and Space Patched Distillation (patch-wise attention-guided distillation).
Problem
Diffusion models are too expensive for resource-constrained deployment, and prior binarization work only binarized weights; fully binarizing weights and activations causes severe degradation or collapse in generation quality.
Approach
Make the XNOR-style activation scale factor a learnable tiny convolution, add cross-timestep feature connections with learnable mixing factors (TBS), and distill binary features from a full-precision teacher by matching patch-wise spatial attention (SPD) rather than with L2 loss.
Key results
- W1A1 BiDM on LDM-4 LSUN-Bedrooms achieves FID 22.74 versus 59.44 for the best baseline (ReSTE).
- On CIFAR-10 DDIM, BiDM raises IS from 4.23 (XNOR) to 5.18 and reduces FID from 113.36 to 81.65.
- Ablation on LSUN-Bedrooms: vanilla FID 106.62, +TBS 35.23, +SPD 40.62, full BiDM 22.74; reported 28.0x storage and 52.7x OPs savings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace XNOR with BiDM full binarization | DDIM · CIFAR-10 32×32 | IS: 4.23 -> 5.18 | Helped |
| replace XNOR with BiDM full binarization | LDM-4 · LSUN-Bedrooms 256×256 | FID: 106.62 -> 22.74 | Helped |
| add TBS (ablation) | LDM-4 · LSUN-Bedrooms 256×256 | FID: 106.62 -> 35.23 | Helped |
| add TBS (ablation) | LDM-4 · LSUN-Bedrooms 256×256 | sFID: 56.81 -> 25.13 | Helped |
| add SPD (ablation) | LDM-4 · LSUN-Bedrooms 256×256 | FID: 106.62 -> 40.62 | Helped |
| make k learnable in XNOR (ablation) | LDM-4 · LSUN-Bedrooms 256×256 | FID: 106.62 -> 57.26 | Helped |
| make k learnable in XNOR (ablation) | LDM-4 · LSUN-Bedrooms 256×256 | Recall: 5.22 -> 5.00 | Hurt |
| add cross-timestep connections instead of none (ablation) | LDM-4 · LSUN-Bedrooms 256×256 | FID: 30.24 -> 24.22 | Helped |
| increase cross-timestep connections from 1 to 8 (ablation) | LDM-4 · LSUN-Bedrooms 256×256 | FID: 24.22 -> 22.74 | Helped |
| replace L2 distillation with SPD patch distillation (ablation) | LDM-4 · LSUN-Bedrooms 256×256 | FID: 26.07 -> 22.74 | Helped |
Surprising findings
- Simply making the XNOR scale convolution learnable reduces FID from 106.62 to 57.26 on LDM-4 LSUN-Bedrooms.
- Generic binarization methods (XNOR++, DoReFa, ReSTE, ReActNet) collapse on diffusion models, with some producing near-zero precision and recall.
Limitations
- BiDM increases training time, since cross-timestep connections require two noise estimations per sample.
- Generation quality remains far from full-precision models (e.g., FID 22.74 vs 2.99 on LSUN-Bedrooms).
- Evaluated only on CIFAR-10, LSUN-Bedrooms, LSUN-Churches and FFHQ with DDIM and LDM models.
Experimental setup
Datasets: CIFAR-10 LSUN-Bedrooms LSUN-Churches FFHQ
Benchmarks: CIFAR-10 LSUN-Bedrooms LSUN-Churches FFHQ
Metrics: IS FID sFID Precision Recall
Baselines: XNOR XNOR++ DoReFa ReActNet ReSTE BBCU EfficientDM
Builds on: XNOR-Net XNOR++ ReActNet BinaryDM DDIM LDM
Closest papers by meaning
- BinaryDM: Accurate Weight Binarization for Efficient Diffusion Models (ICLR 2025 · similarity 0.93)
- Optimizing Quantized Diffusion Models via Distillation with Cross-Timestep Error Correction (AAAI 2025 · similarity 0.90)
- MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models (AAAI 2025 · similarity 0.90)
- Binarized Diffusion Model for Image Super-Resolution (NeurIPS 2024 · similarity 0.90)
- TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Models (AAAI 2025 · similarity 0.90)
- BitsFusion: 1.99 bits Weight Quantization of Diffusion Model (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.