Research map / Image generation / Fast diffusion sampling
Split Gibbs Discrete Diffusion Posterior Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SGDD, a plug-and-play discrete diffusion posterior sampler built on split Gibbs sampling with a Hamming-distance regularizer, enabling reward-guided generation and inverse problems over discrete states.
Problem
Posterior sampling with discrete diffusion models for inverse problems and reward-guided generation, where likelihoods lack useful gradients on discrete state spaces.
Approach
Generalizes split Gibbs sampling to discrete states by using a Hamming-distance-based regularization potential, so the prior step becomes a partial discrete diffusion sampler started at noise level eta and the likelihood step is an MCMC (Metropolis-Hastings) move that needs no gradients of the guidance term; an annealed eta schedule drives convergence to the posterior.
Key results
- On MNIST XOR inverse problem, SGDD reaches 20.17 dB PSNR and 91.2% accuracy vs 11.81 dB and 51.4% for SVDD-PM.
- On guided DNA enhancer design, SGDD (beta=50) achieves median predicted activity 9.14 vs 6.44 for DRAKES w/o KL, a 42% gain over the prior state of the art.
- On monophonic music infilling at 40% masking, SGDD reaches Hellinger 0.126 vs 0.465 for SVDD-PM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SGDD instead of DPS | Synthetic posterior sampling · Synthetic D=2 | Hellinger distance: 0.182 -> 0.149 | Helped |
| Use SGDD instead of DPS | Synthetic posterior sampling · Synthetic D=5 | Hellinger distance: 0.345 -> 0.214 | Helped |
| Use SGDD instead of DPS | Synthetic posterior sampling · Synthetic D=10 | Hellinger distance: 0.410 -> 0.334 | Helped |
| Use SGDD instead of SVDD-PM (M=20) | MNIST inverse problems · MNIST XOR | PSNR: 11.81_{\pm 2.54} -> 20.17_{\pm 3.47} | Helped |
| Use SGDD instead of SVDD-PM (M=20) | MNIST inverse problems · MNIST AND | PSNR: 10.04_{\pm 1.49} -> 17.25_{\pm 3.82} | Helped |
| Use SGDD instead of DRAKES w/o KL | DNA enhancer generation · DNA enhancers | Median predicted activity: 6.44 -> 9.14 | Helped |
| Use SGDD instead of DRAKES w/ KL | DNA enhancer generation · DNA enhancers | ATAC accuracy: 92.5 -> 93.0 | Helped |
| Use SGDD instead of SVDD-PM | Music infilling · Lakh monophonic music infilling, γ=40% | Hellinger distance: 0.465_{\pm 0.129} -> 0.126_{\pm 0.098} | Helped |
| Use SGDD instead of SVDD-PM | Music infilling · Lakh monophonic music infilling, γ=60% | Hellinger distance: 0.529_{\pm 0.137} -> 0.244_{\pm 0.164} | Helped |
| Increase β from 30 to 50 (ablation) | SGDD DNA enhancer generation · DNA enhancers | Median predicted activity: 8.82 -> 9.14 | Helped |
| Increase β from 30 to 50 (ablation) | SGDD DNA enhancer generation · DNA enhancers | Average log-likelihood: -246_{\pm 41} -> -261_{\pm 29} | Hurt |
| Replace fixed noise with an annealing noise schedule (ablation) | SGDD on MNIST AND · MNIST AND | PSNR | Helped |
Surprising findings
- Fixed small noise levels converge slowly and give poor solutions, motivating an annealed eta schedule.
Limitations
- Synthetic experiments show slight deviation from the true posterior as dimensionality grows.
- Theoretical guarantee relies on assumptions such as bounded probability ratio and score estimation error epsilon, and holds for fixed eta in the limit.
- Fixed small eta gives ill-conditioned transition rates and slow convergence, requiring an annealing schedule.
Experimental setup
Datasets: MNIST MPRA DNA enhancer dataset (~700k sequences) Lakh pianoroll dataset FFHQ 256
Benchmarks: synthetic discretized Gaussian posterior MPRA DNA enhancer design discretized MNIST XOR/AND inverse problems Lakh monophonic music infilling
Metrics: Hellinger distance total variation predicted activity ATAC accuracy log-likelihood PSNR classifier accuracy outlier notes proportion
Baselines: DPS (discrete) SVDD-PM SMC CG TDS FK-MAX DRAKES
Builds on: split Gibbs sampler SEDD DPS SVDD
Closest papers by meaning
- Divide-and-Conquer Posterior Sampling for Denoising Diffusion priors (NeurIPS 2024 · similarity 0.91)
- A Mixture-Based Framework for Guiding Diffusion Models (ICML 2025 · similarity 0.90)
- Provable Posterior Sampling with Denoising Oracles via Tilted Transport (NeurIPS 2024 · similarity 0.89)
- SITCOM: Step-wise Triple-Consistent Diffusion Sampling For Inverse Problems (ICML 2025 · similarity 0.89)
- Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.89)
- Provably Robust Score-Based Diffusion Posterior Sampling for Plug-and-Play Image Reconstruction (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.