Research map / Probabilistic and scientific modeling / Variational and generative sampling
Diffusive Gibbs Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Diffusive Gibbs Sampling (DiGS), which Gibbs-samples between a Gaussian-convolved noisy distribution and the tractable denoising posterior, with a Metropolis-within-Gibbs initialization step, to sample multi-modal energy-based targets.
Problem
Standard MCMC (MALA, HMC) and parallel tempering mix poorly across distant or disconnected modes of unnormalized multi-modal distributions, as in Bayesian inference and molecular dynamics.
Approach
Use a Gaussian convolution to create an auxiliary noisy variable that bridges isolated modes, alternate Gibbs draws between the noisy kernel and the tractable denoising posterior, and initialize the denoising sampler via an MH step on a linear Gaussian proposal centered at the scaled noisy sample.
Key results
- On MoG-40, DiGS reaches MMD (4.57±1.10)×10^-4 versus 1.89×10^-2 for PT and about 1.7 for MALA and HMC, with equal 10^7 energy evaluations.
- On the BNN posterior, DiGS achieves test NLL 0.189±0.002 versus 0.241 for PT and 0.315 for HMC.
- On alanine dipeptide, DiGS beats MD PT on KL divergence (Ramachandran 4.3e-3 vs 5.3e-3) using 23× fewer energy evaluations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DiGS with Gaussian convolution and Metropolis-within-Gibbs initialization instead of MALA | MoG-40 sampling · MoG-40 | MMD: 1.73\pm0.12 -> (4.57\pm1.10)\times 10^{-4} | Helped |
| Use DiGS with Gaussian convolution and Metropolis-within-Gibbs initialization instead of PT | MoG-40 sampling · MoG-40 | MAE (%): 7.32\pm1.85 -> 0.75\pm0.19 | Helped |
| Use DiGS sampling instead of PT | Bayesian neural network posterior sampling · BNN test | NLL: 0.241\pm0.005 -> 0.189\pm0.002 | Helped |
| Use DiGS sampling instead of MD PT | Alanine dipeptide configuration sampling · Alanine dipeptide, Ramachandran | KL divergence: 5.3{\times} 10^{-3} -> 4.3{\times} 10^{-3} | Helped |
| Use DiGS sampling instead of PT | 200-dimensional MoG sampling · 200-dimensional MoG | MMD: 0.094\pm0.014 -> 0.027\pm0.010 | Helped |
| Add Metropolis-Hastings initialization instead of initializing at the previous state (ablation) | DiGS denoising step · Unequally weighted MoG | MMD: 0.15\pm 0.01 -> 0.03\pm 0.01 | Helped |
| Add Metropolis-Hastings initialization instead of initializing at the scaled noisy sample (ablation) | DiGS denoising step · Unequally weighted MoG | MMD: 0.92\pm 0.04 -> 0.03\pm 0.01 | Helped |
| Initialize denoising at the scaled noisy sample instead of the previous state (ablation) | DiGS denoising step · Unequally weighted MoG | MMD: 0.15\pm 0.01 -> 0.92\pm 0.04 | Hurt |
| Use a VP noise schedule with T>2 noise levels (ablation) | DiGS · MoG | MMD | Helped |
| Use Metropolis-within-Gibbs initialization (ablation) | DiGS denoising step · Denoising initialization | acceptance rate: around $0.15$ | Hurt |
Surprising findings
- The Metropolis-within-Gibbs acceptance rate was only around 0.15, attributed to random-walk behavior of the linear Gaussian proposal.
- Standard parallel tempering fails to escape the central mode on the Mixture of Deltas problem even with large temperatures, requiring 20 chains and DEO to work.
Limitations
- Metropolis-within-Gibbs acceptance rate is low (about 0.15) due to random-walk linear Gaussian proposal.
- Hyperparameters alpha and sigma must be chosen within a range that depends on the unknown target; multi-level scheduling partly mitigates this.
- No general convergence analysis for the multi-level noise schedule.
Experimental setup
Datasets: mixture of 40 Gaussians Bayesian neural network synthetic regression data alanine dipeptide energy function
Benchmarks: MoG-40 200-dimensional mixture of Gaussians Bayesian neural network test NLL alanine dipeptide Ramachandran plots
Metrics: MMD MAE of expectation (%) NLL KL divergence energy evaluations
Baselines: MALA HMC parallel tempering MAP RDMC MD PT
Builds on: Metropolis-adjusted Langevin algorithm Hamiltonian Monte Carlo parallel tempering proximal sampler reverse diffusion monte carlo
Closest papers by meaning
- Split Gibbs Discrete Diffusion Posterior Sampling (NeurIPS 2025 · similarity 0.88)
- Composing Diffusion Priors with Explicit Physical Context via Generative Gibbs Sampling (NeurIPS 2026 · similarity 0.88)
- Sampling from multi-modal distributions with polynomial query complexity in fixed dimension via reverse diffusion (NeurIPS 2025 · similarity 0.88)
- On scalable and efficient training of diffusion samplers (NeurIPS 2025 · similarity 0.88)
- Progressive Inference-Time Annealing of Diffusion Models for Sampling from Boltzmann Densities (NeurIPS 2025 · similarity 0.87)
- A Mixture-Based Framework for Guiding Diffusion Models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.