Research map / Image generation / Fast diffusion sampling
Particle Denoising Diffusion Sampler
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Particle Denoising Diffusion Sampler (PDDS), an SMC scheme that uses guided denoising-diffusion-style backward kernels with a novel score matching (NSM) loss to learn potentials, giving consistent normalizing constant and sample estimates for unnormalized densities.
Problem
Sampling from unnormalized densities and estimating their normalizing constants with diffusion-based methods, which are biased due to approximate guidance potentials and cannot provide consistent estimates; existing diffusion samplers also rely on reverse-KL objectives or poorly scaling importance s
Approach
Use the guidance approximation of the noising-diffusion marginals as an initial potential, run an SMC particle scheme backward in time with a discretized denoising proposal and importance weights (correcting the approximation exactly in the limit), and learn improved potentials by a novel score matching loss (NSM) that uses the particle output and avoids variance blow-up at small times.
Key results
- On the posterior tasks PDDS-MCMC is best in bias and variance everywhere except 4 steps on Sonar, where CRAFT is best; both PDDS variants uniformly outperform PIS and DDS.
- On GMM in 20 dimensions, PDDS-MCMC significantly outperforms CRAFT in bias and variance and in entropy-regularized W2 distance, especially in low-step regimes.
- The simple guidance approximation gives very poor normalizing constant estimates, but iterating the learned NSM potentials considerably improves them, surpassing PIS and DDS after about 7x10^7 density evaluations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add optional MCMC steps (ablation) | PDDS · Sonar | normalizing-constant estimation bias and variance | Helped |
| Use PDDS-MCMC instead of CRAFT | GMM, 20 dimensions · GMM | normalizing-constant estimation bias and variance | Helped |
| Use PDDS-MCMC instead of CRAFT | GMM, 20 dimensions · GMM | entropy-regularized Wasserstein-2 distance | Helped |
| Use PDDS instead of PIS or DDS | PDDS · Mixture | sample mode coverage | Helped |
| Use PDDS instead of PIS or DDS | PDDS · Normalizing-constant estimation tasks | normalizing-constant estimation | Helped |
| Use PDDS instead of CRAFT | PDDS · Sonar and LGCP | normalizing-constant estimation | Mixed |
| Use CRAFT instead of PDDS | Normalizing-constant estimation methods · Funnel | normalizing-constant estimation | Hurt |
| Use PDDS instead of CRAFT | Normalizing-constant estimation methods · Mixture | normalizing-constant estimation | Helped |
| Replace the simple potential approximation with an iteratively learned potential (ablation) | PDDS · Sonar | normalizing-constant estimation | Helped |
| Replace the simple potential approximation with an iteratively learned potential (ablation) | PDDS · Sonar | effective sample size | Helped |
| Train the potential approximation iteratively (ablation) | PDDS · Sonar | normalizing-constant estimation relative to PIS and DDS: around 7×10^7 density evaluations | Helped |
| Use standard denoising score matching instead of novel score matching (ablation) | Score-matching potential learning · Gaussian target, continuous-time limit | score-matching loss finiteness | Hurt |
Surprising findings
- The simple guidance approximation of the potential gives very poor normalizing constant estimates, and the proposed method initially falls below PIS and DDS before overtaking them.
Limitations
- Practically relies on g_0 being a well-behaved potential function, requiring a variational reparameterization of the target.
- Finite particle counts give only asymptotic consistency; theory for adaptive/systematic resampling is lacking.
- Multinomial resampling leads to particle collapse as discretization becomes fine, so sorted stratified resampling is needed for theory.
Experimental setup
Datasets: Mixture (2D Gaussian mixture) Funnel (10D) Sonar (61D logistic regression posterior) LGCP (Pines dataset, 1600-point grid) GMM (40-mode Gaussian mixture, multiple dimensions)
Benchmarks: Mixture Funnel Sonar LGCP GMM
Metrics: log normalizing constant estimate bias and variance entropy-regularized Wasserstein-2 distance effective sample size
Baselines: SMC with HMC kernels CRAFT PIS DDS
Builds on: denoising diffusion models sequential monte carlo samplers Denoising Diffusion Sampler (DDS) Path Integral Sampler (PIS) CRAFT
Closest papers by meaning
- Assessing the quality of denoising diffusion models in Wasserstein distance: noisy score and optimal bounds (NeurIPS 2025 · similarity 0.90)
- Denoising Diffusion Bridge Models (ICLR 2024 · similarity 0.90)
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions (ICLR 2025 · similarity 0.90)
- Interpreting and Improving Diffusion Models from an Optimization Perspective (ICML 2024 · similarity 0.90)
- Distributional Diffusion Models with Scoring Rules (ICML 2025 · similarity 0.90)
- Beyond Scores: Proximal Diffusion Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.