Research map / Probabilistic and scientific modeling / Variational and generative sampling
Sampling by averaging: A multiscale approach to score estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes training-free score-based samplers (MultALMC, MultCDiff) that estimate diffusion-path scores via a fast-slow SDE system with stochastic averaging, avoiding nested MCMC loops.
Problem
Sampling from unnormalised target densities with score-based diffusion paths requires estimating intermediate scores, which normally needs learned networks or expensive nested MCMC loops.
Approach
Run a slow annealed/controlled diffusion alongside a faster auxiliary fast process targeting the conditional distribution rho_{t,x}, so stochastic averaging yields the intermediate score without an inner MCMC loop or training; includes underdamped variants and heavy-tailed (Student-t) extensions.
Key results
- MultCDiff achieves the lowest entropy-regularised W2 on 8-MoG (0.62 ± 0.10 vs. 1.01 for next best RDMC) and 50-dim DW (13.98 ± 0.56 vs. 16.13 for PT, MultCDiff 13.98 per table).
- MultALMC achieves best Rings W2 (0.18 ± 0.02) and best Sonar log-likelihood (-109.05 ± 0.13) among methods reported.
- Authors report their methods outperform all baselines across benchmarks at equal energy-evaluation budgets, with overdamped variants needing roughly an order of magnitude more steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MultALMC instead of the strongest listed baseline | MultALMC · 8-MoG (d=2) | entropy-regularised Wasserstein-2 distance: 1.01 ± 0.21 -> 0.65 ± 0.07 | Helped |
| Use MultCDiff instead of the strongest listed baseline | MultCDiff · 40-MoG (d=50) | entropy-regularised Wasserstein-2 distance: 21.04 ± 0.55 -> 20.13 ± 0.59 | Helped |
| Use multiscale controlled diffusion instead of annealed Langevin dynamics | MultCDiff vs MultALMC · Ionosphere (d=35) | predictive posterior log-likelihood: -86.85 ± 0.09 -> -86.33 ± 0.10 | Helped |
| Use multiscale controlled diffusion instead of annealed Langevin dynamics | MultCDiff vs MultALMC · Sonar (d=61) | predictive posterior log-likelihood: -109.05 ± 0.13 -> -109.60 ± 0.21 | Hurt |
| Use underdamped rather than overdamped dynamics | MultALMC and MultCDiff · Sampling benchmarks | steps to comparable performance: null | Helped |
Surprising findings
- Overdamped variants require approximately an order of magnitude more steps than underdamped ones to reach comparable performance.
Limitations
- Theoretical guarantees rely on strong log-concavity assumptions on the target that are restrictive.
- Hyperparameters (step size, scale separation epsilon, friction Gamma) require manual tuning.
- Extension of controlled diffusions to heavy-tailed targets is not addressed.
Experimental setup
Datasets: Mixture of Gaussians Mixture of Student's t Rings Funnel Double well potential Ionosphere Sonar stochastic Allen-Cahn phi^4 model
Benchmarks: 8-MoG 40-MoG Rings Funnel 5-dim DW 10-dim DW 50-dim DW Ionosphere Sonar phi^4 model
Metrics: entropy-regularised Wasserstein-2 distance sliced Kolmogorov-Smirnov distance average predictive posterior log-likelihood estimated mode weight ratio
Baselines: SMC AIS ULMC PT DiGS RDMC SLIPS
Builds on: annealed Langevin dynamics stochastic averaging theory SROCK proximal samplers diffusive Gibbs sampling
Closest papers by meaning
- Diffusion Path Samplers via Sequential Monte Carlo (NeurIPS 2026 · similarity 0.92)
- Reverse Diffusion Sequential Monte Carlo Samplers (NeurIPS 2025 · similarity 0.88)
- Sequential Controlled Langevin Diffusions (ICLR 2025 · similarity 0.88)
- Accelerating Convergence of Score-Based Diffusion Models, Provably (ICML 2024 · similarity 0.88)
- Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis (ICLR 2025 · similarity 0.87)
- Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplers (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.