Research map / Probabilistic and scientific modeling / Variational and generative sampling
Reverse Diffusion Monte Carlo
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes reverse diffusion Monte Carlo (rdMC), a non-parametric sampler that runs the reverse SDE of an OU process and estimates scores as posterior means via importance sampling or ULA, with complexity analysis versus Langevin MCMC.
Problem
Sampling from unnormalized multi-modal and heavy-tailed target distributions, where Langevin MCMC mixes slowly due to poor log-Sobolev constants and isoperimetry.
Approach
Rewrites the score of the noised OU marginals as the posterior mean of the clean variable under a tilted distribution, replacing learned score networks with non-parametric Monte Carlo mean estimation (importance sampling or inner ULA) inside a reverse-SDE sampler.
Key results
- On Gaussian mixtures with increasing mode separation r, rdMC keeps converging in MMD where LMC fails to converge within a limited time, and ULMC only partially alleviates this.
- Theory shows rdMC's gradient complexity for the Gaussian mixture can avoid the exponential dependence on squared mode distance ||y||^2 that MCMC lower bounds predict, reducing it by a factor e^{-2T}=2/3 in the exponent with T=0.5 log(3/2).
- For heavy-tailed potentials exp(-(||x||^2+1)^a) with a<0.5, the rdMC complexity has lower dimension dependence than the weak-Poincaré ULA bound d^{4/a+3} when a<=4/15.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use rdMC with importance-sampling score estimation instead of Langevin-style MCMC | Gaussian mixture sampling · Gaussian mixture model | iteration complexity: \Omega(\exp(\Vert y\Vert^2/8) vs. \tilde{O}(\epsilon^{-2}) | Helped |
| Set T = \frac{1}{2}\log \frac{3}{2} for ULA initialization of p_T | rdMC · Gaussian mixture model | initialization iterations: O(e^{y^2}) -> O(e^{\frac{2}{3}y^2}) | Helped |
| Use rdMC instead of ULA for sub-linear-tail potentials | Sampling distributions with sub-linear tails · Sub-linear-tail potentials | dimension dependence | Helped |
Limitations
- Importance-sampling score estimator sample size can scale exponentially with dimension because the normalizing constant Z can be as small as exp(-d).
- Theoretical complexity bounds for heavy-tailed case are very large (e.g., exp(epsilon^{-16n})), limiting practical implications.
- Experiments are limited to low-dimensional Gaussian mixture examples.
Experimental setup
Benchmarks: Gaussian mixture models
Metrics: maximum mean discrepancy gradient complexity
Baselines: Langevin Monte Carlo (LMC) Underdamped Langevin Monte Carlo (ULMC)
Builds on: Langevin dynamics denoising diffusion probabilistic models Schrödinger-Föllmer sampler
Closest papers by meaning
- Reverse Diffusion Sequential Monte Carlo Samplers (NeurIPS 2025 · similarity 0.91)
- Diffusion Path Samplers via Sequential Monte Carlo (NeurIPS 2026 · similarity 0.89)
- Reverse Transition Kernel: A Flexible Framework to Accelerate Diffusion Inference (NeurIPS 2024 · similarity 0.88)
- Distributional Diffusion Models with Scoring Rules (ICML 2025 · similarity 0.88)
- Sampling from multi-modal distributions with polynomial query complexity in fixed dimension via reverse diffusion (NeurIPS 2025 · similarity 0.88)
- Particle Denoising Diffusion Sampler (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.