Research map / Probabilistic and scientific modeling / Variational and generative sampling
Reverse Diffusion Sequential Monte Carlo Samplers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Reverse Diffusion SMC (RDSMC), a training-free SMC sampler that wraps reverse-diffusion Monte Carlo score proposals with exact-approximation intermediate targets built from AIS marginal estimates, giving consistent samples and unbiased normalization constant estimates.
Problem
Sampling from unnormalized target distributions with diffusion-based samplers, whose discretization and score-estimation errors accumulate and bias the output, and whose trained alternatives are computationally costly.
Approach
Treat MC-score-based reverse diffusion as an SMC proposal and correct its bias with intermediate targets built from unbiased marginal estimates (byproducts of AIS score estimation), which are exact approximations of the optimal extended-target marginals, requiring no training.
Key results
- On Rings (d=2), RDSMC achieves the lowest log Z bias (0.03 ± 0.00) among methods that estimate Z, versus 0.05 for AIS and SMC.
- On bi-modal Gaussian mixtures in high dimensions, AIS and SMC collapse to a single mode with about 0.4 weight-ratio bias, while RDSMC reaches 0.09 and 0.08 for d=32 and d=64.
- On Bayesian logistic regression, RDSMC is competitive with SLIPS (e.g., -101.52 vs -102.39 test LL on Sonar) and far above RDSMC (Proposal), but SLIPS is best overall.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add intermediate SMC weighting and resampling to the reverse-diffusion proposal (ablation) | RDSMC · Bayesian logistic regression (Credit, Cancer, Ionosphere, Sonar) | Test log-likelihood: -606.03 $\pm$ 1.26 -> -124.00 $\pm$ 1.96; -246.86 $\pm$ 0.97 -> -62.23 $\pm$ 1.96; -92.62 $\pm$ 0.04 -> -87.72 $\pm$ 1.7 | Helped |
| add intermediate SMC resampling instead of using a final importance-sampling correction (ablation) | RDSMC · Funnel (d=10) | Sliced KSD: 0.33 $\pm$ 0.03 -> 0.11 $\pm$ 0.00 | Helped |
| add intermediate SMC resampling instead of using a final importance-sampling correction (ablation) | RDSMC · Funnel (d=10) | log Z bias: 1.61 $\pm$ 0.14 -> 0.28 $\pm$ 0.10 | Helped |
| add intermediate SMC resampling instead of using a final importance-sampling correction (ablation) | RDSMC · Rings (d=2) | Radius TVD: 0.15 $\pm$ 0.01 -> 0.13 $\pm$ 0.01 | Hurt |
| add intermediate SMC resampling instead of using a final importance-sampling correction (ablation) | RDSMC · Rings (d=2) | log Z bias: 0.02 $\pm$ 0.01 -> 0.03 $\pm$ 0.00 | Hurt |
| add intermediate SMC resampling instead of using a final importance-sampling correction (ablation) | RDSMC · Bayesian logistic regression (Credit, Cancer) | Test log-likelihood: -144.38 $\pm$ 4.63 -> -124.00 $\pm$ 1.96; -82.47 $\pm$ 7.78 -> -62.23 $\pm$ 1.96 | Helped |
| add intermediate SMC resampling instead of using a final importance-sampling correction (ablation) | RDSMC · Bayesian logistic regression (Ionosphere) | Test log-likelihood: -84.90 $\pm$ 2.84 -> -87.72 $\pm$ 1.75 | Hurt |
| add intermediate SMC resampling instead of using a final importance-sampling correction (ablation) | RDSMC · Bayesian logistic regression (Sonar) | Test log-likelihood: -110.57 $\pm$ 4.54 -> -101.52 $\pm$ 1.84 | Helped |
| use intermediate SMC resampling rather than the uncorrected reverse-diffusion proposal (ablation) | RDSMC · Rings (d=2) | Radius TVD: 0.09 $\pm$ 0.00 -> 0.13 $\pm$ 0.01 | Hurt |
| use intermediate SMC resampling rather than the uncorrected reverse-diffusion proposal (ablation) | RDSMC · Rings (d=2) | log Z bias | Helped |
| use intermediate SMC resampling rather than the uncorrected reverse-diffusion proposal (ablation) | RDSMC · Funnel (d=10) | Sliced KSD: 0.32 $\pm$ 0.03 -> 0.11 $\pm$ 0.00 | Helped |
| use intermediate SMC resampling rather than the uncorrected reverse-diffusion proposal (ablation) | RDSMC · Bayesian logistic regression (Credit, Cancer, Ionosphere, Sonar) | Test log-likelihood: -606.03 $\pm$ 1.26 -> -124.00 $\pm$ 1.96; -246.86 $\pm$ 0.97 -> -62.23 $\pm$ 1.96; -92.62 $\pm$ 0.04 -> -87.72 $\pm$ 1.7 | Helped |
Surprising findings
- The proposal-only variant RDSMC (Proposal) achieves the lowest radius TVD on Rings, beating the full RDSMC, although its weights are poorly calibrated.
- On Funnel, SLIPS beats RDSMC on sliced KSD, and RDSMC (IS) has much worse log Z bias (1.61) than full RDSMC (0.28).
Limitations
- Auxiliary variance from nested SMC structure leads to higher variance than baselines on logistic regression.
- Mode collapse in high-dimensional Gaussian mixtures.
- Hyperparameter tuning relies on oracle metrics or validation data.
Experimental setup
Datasets: Bi-modal Gaussian mixtures Rings Funnel Credit Cancer Ionosphere Sonar
Benchmarks: Bi-modal Gaussian mixtures Rings Funnel Bayesian logistic regression (Credit, Cancer, Ionosphere, Sonar)
Metrics: weight ratio estimation bias log normalization constant bias radius TVD sliced KSD test log-likelihood
Baselines: AIS SMC SMS RDMC SLIPS RDSMC (IS) RDSMC (Proposal)
Builds on: Diffusion MC samplers (RDMC) SLIPS nested sequential Monte Carlo annealed importance sampling diffusion-based samplers with SMC correction
Closest papers by meaning
- Diffusion Path Samplers via Sequential Monte Carlo (NeurIPS 2026 · similarity 0.92)
- Reverse Diffusion Monte Carlo (ICLR 2024 · similarity 0.91)
- Sequential Controlled Langevin Diffusions (ICLR 2025 · similarity 0.91)
- Zeroth-Order Sampling Methods for Non-Log-Concave Distributions: Alleviating Metastability by Denoising Diffusion (NeurIPS 2024 · similarity 0.90)
- Distributional Diffusion Models with Scoring Rules (ICML 2025 · similarity 0.89)
- Tuning Sequential Monte Carlo Samplers via Greedy Incremental Divergence Minimization (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.