Research map / Image generation / Fast diffusion sampling
Improved off-policy training of diffusion samplers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Benchmarks diffusion-structured samplers (GFlowNets vs. PIS/DDS/DIS/SMC/GGNS) under unified settings and proposes local search with a replay buffer (parallel MALA) for off-policy exploration in continuous GFlowNets.
Problem
Training diffusion models to sample from unnormalized densities or energies is hard due to poor exploration and credit assignment, and prior benchmarks are inconsistent and irreproducible.
Approach
Unified library for diffusion-structured samplers with off-policy GFlowNet training, plus a new exploration strategy: parallel MALA local search in target space whose accepted samples fill a replay buffer used to train the sampler alternately with forward trajectories.
Key results
- TB + Expl. + LP + LS reaches Δ log Z of 0.190±0.013 on 25GMM and 0.007±0.011 on the RW estimate, versus 1.176±0.109 for TB alone.
- On Manywell (d=32), TB + Expl. + LP + LS achieves Δ log Z_RW 0.07±0.17 versus 2.68 for TB + Expl., with comparable W2 (5.33 vs 6.15 for VarGrad + Expl.).
- FL-SubTB gives no consistent benefit over TB or VarGrad; the reported gains of FL-SubTB + LP in prior work did not reproduce under matched settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add local search with a replay buffer to TB + exploration | TB + Expl. · 25GMM | Δ log Z: 0.560±0.302 -> 0.171±0.013 | Helped |
| Add local search with a replay buffer to TB + exploration | TB + Expl. · 25GMM | Δ log ZRW: 0.422±0.320 -> 0.004±0.011 | Helped |
| Add local search with a replay buffer to TB + exploration | TB + Expl. · Funnel | Δ log Z: 0.749±0.015 -> 0.653±0.025 | Mixed |
| Add local search with a replay buffer to TB + exploration | TB + Expl. · Manywell | Δ log ZRW: 2.68±0.06 -> 0.19±0.29 | Helped |
| Add Langevin parametrization to TB + exploration | TB + Expl. · Manywell | Δ log ZRW: 2.68±0.06 -> 1.06±1.11 | Helped |
| Replace TB with FL-SubTB | diffusion GFlowNet · 25GMM | Δ log Z: 1.176±0.109 -> 1.127±0.010 | No clear effect |
| Replace TB with VarGrad | diffusion GFlowNet with exploration · Manywell | Δ log Z: 4.01±0.05 -> 4.01±0.05 | No clear effect |
| Add Langevin parametrization to the sampler | TB + Expl. · Manywell | Δ log Z: 4.01±0.05 -> 7.46±1.74 | Hurt |
| Add local search to TB + Expl. + LP | TB + Expl. + LP · Manywell | Δ log ZRW: 1.06±1.11 -> 0.07±0.17 | Helped |
| Add Langevin parametrization to TB | TB · VAE latent posterior (MNIST test set) | log Z estimate: −162.73±35.55 -> −112.45±0.671 | Helped |
| Add local search to VarGrad + Expl. + LP | VarGrad + Expl. + LP · VAE latent posterior (MNIST test set) | log ZRW estimate: −46.574±0.736 -> −46.245±0.543 | Helped |
| Increase replay buffer capacity from 30,000 to 600,000 (ablation) | TB + Expl. + LS · Manywell | Δ log ZRW: 2.73±0.15 -> 0.19±0.29 | Helped |
Surprising findings
- Reported benefits of FL-SubTB over TB in prior work (Zhang and Chen-style comparison) did not hold when Langevin parametrization was matched, and the published results were not reproducible even with the authors' code.
- Prior benchmarks disagree on the Funnel definition (variance 1 vs 9) and the LGCP ground-truth log Z (512.6 vs 501.8).
Limitations
- Continuous-time limit (T to infinity) of the compared algorithms is not analyzed.
- Learned forward variance and non-Gaussian transitions are only preliminarily explored; coarse discretizations (few steps) are less accurate.
- VAE conditional sampling: TB-based GFlowNets remain worse than PIS in likelihood estimation, and local search with TB did not help.
Experimental setup
Datasets: 25GMM Funnel Manywell LGCP MNIST
Benchmarks: 25GMM Funnel Manywell LGCP VAE latent posterior (MNIST)
Metrics: log-partition function error (Δ log Z) importance-weighted log Z estimate (log Z_RW) 2-Wasserstein distance log Z estimate on VAE test set
Baselines: SMC GGNS DIS DDS PIS TB VarGrad FL-SubTB
Builds on: continuous GFlowNets path integral sampler denoising diffusion sampler subtrajectory balance Metropolis-adjusted Langevin
Closest papers by meaning
- Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.91)
- On scalable and efficient training of diffusion samplers (NeurIPS 2025 · similarity 0.90)
- Outsourced Diffusion Sampling: Efficient Posterior Inference in Latent Spaces of Generative Models (ICML 2025 · similarity 0.89)
- Hierarchical Variational Policies for Reward-Guided Diffusion (NeurIPS 2026 · similarity 0.89)
- Underdamped Diffusion Bridges with Applications to Sampling (ICLR 2025 · similarity 0.89)
- The Emergence of Reproducibility and Consistency in Diffusion Models (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.