Research map / Image generation / Fast diffusion sampling

Efficient Denoising Diffusion via Probabilistic Masking

ICML 2024 · Image generation › Fast diffusion sampling · approach: Fast diffusion ODE solvers

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EDDPM, which learns Bernoulli step-masks jointly with diffusion weights under a sparsity constraint so that redundant denoising steps are skipped during training, cutting inference steps for time-series imputation and CIFAR-10 generation.

Problem

Diffusion models need many denoising steps at inference; existing acceleration relies on handcrafted skipping rules or decoupled training/sampling schedules that can discard important steps.

Approach

Assign each diffusion step a Bernoulli mask with learnable probability s_t, relax the discrete step-selection into a continuous problem optimized with policy gradients and a projected L1-type sparsity constraint, so the training and sampling schedules coincide and the masks become near-deterministic.

Key results

• On CIFAR-10 with 5 denoising steps, EDDPM reaches 4.89 FID, versus 41.6 for DDIM at 5 steps.
• On Healthcare and Air-quality imputation, EDDPM with 20% of steps matches or beats the 100%-step DDPM baseline in RMSE (e.g., Air-quality 18.24 vs 31.34 at 40% missing).
• Fine-tuning a pretrained CSDI for one epoch (about 1 min) to select steps gives 9.89 MAE at 40% steps, comparable to training from scratch (9.50-9.90 at 20-40%).

Measured effects reported in the paper

Surprising findings

• Removing a single step from CSDI often has negligible or even positive effect on sample quality (Figure 1).
• Training from scratch with EDDPM reduces MAE at low step counts compared to DDPM trained for the same time (e.g., 15.13 vs 55.08 at 2% steps), suggesting the masked training converges faster.

Limitations

• Gains on CIFAR-10 are reported only against a learning-free and learning-based baseline set with differing base models; the TDAS NCSN++ result is better than EDDPM due to a different base model.
• Time-series baselines were re-implemented in the authors' codebase and evaluated only on two datasets.
• Combination with consistency models is left for future work.

Experimental setup

Datasets: CIFAR-10 Healthcare Air-quality

Benchmarks: CIFAR-10 Healthcare Air-quality

Metrics: FID RMSE MAE CRPS

Baselines: DDPM DDIM Analytic-DPM SN-DDPM NPR-DDIM PNDM GGDM DPM-solver GENIE TDAS CSDI

Builds on: DDPM DDIM CSDI Analytic-DPM

Closest papers by meaning

• O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.91)
• Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.91)
• Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusion (NeurIPS 2025 · similarity 0.90)
• Simplified and Generalized Masked Diffusion for Discrete Data (NeurIPS 2024 · similarity 0.90)
• AccuQuant: Simulating Multiple Denoising Steps for Quantizing Diffusion Models (NeurIPS 2025 · similarity 0.90)
• Test-Time Scaling of Diffusion Models via Noise Trajectory Search (NeurIPS 2025 · similarity 0.90)