Research map / Image generation / Fast diffusion sampling
Improved sampling via learned diffusions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Unifies diffusion-based sampling methods (SB, DIS, PIS, DDS) via path-space time-reversal and proposes a log-variance divergence loss that avoids differentiating through SDE solvers and improves mode coverage and accuracy.
Problem
Sampling from unnormalized target densities (no samples available) using learned controlled diffusion processes; existing reverse-KL objectives suffer from mode collapse, training instability and need to differentiate through SDE solvers.
Approach
Formulate diffusion-based sampling as a time-reversal problem on path space measures with arbitrary divergences; show SB, DIS, PIS, DDS are special cases; propose the log-variance divergence between path measures, whose Monte Carlo loss needs no backprop through the SDE solver and has zero-variance gradients at the optimum.
Key results
- On the GMM (d=2), PIS with the log-variance loss reduces the log-normalizing-constant error from 1.094 (KL) to 0.046 and raises ESS from 0.0051 to 0.9093.
- On the 5-dimensional double well, DIS with log-variance loss reduces Delta log Z from 1.462 (KL) to 0.375 and W2 from 1.175 to 0.120.
- Log-variance loss improves PIS and DIS across benchmarks, with only small or insignificant differences where KL is better (e.g., Funnel and high-dimensional DW d=50).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use log-variance loss instead of KL loss (ablation) | PIS · GMM (d=2) | ESS: 0.0051 -> 0.9093 | Helped |
| Use log-variance loss instead of KL loss (ablation) | DIS · GMM (d=2) | ΔlogZ: 1.551 -> 0.056 | Helped |
| Use log-variance loss instead of KL loss (ablation) | DIS · GMM (d=2) | Δstd: 2.522 -> 0.004 | Helped |
| Use log-variance loss instead of KL loss (ablation) | PIS · Funnel (d=10) | ESS: 0.1333 -> 0.0746 | Hurt |
| Use log-variance loss instead of KL loss (ablation) | PIS · Funnel (d=10) | ΔlogZ: 0.288 -> 0.277 | Helped |
| Use log-variance loss instead of KL loss (ablation) | DIS · Funnel (d=10) | ESS: 0.1383 -> 0.2261 | Helped |
| Use log-variance loss instead of KL loss (ablation) | PIS · DW (d=5, m=5, δ=4) | ESS: 0.0004 -> 0.6744 | Helped |
| Use log-variance loss instead of KL loss (ablation) | DIS · DW (d=5, m=5, δ=4) | W²γ: 1.175 -> 0.120 | Helped |
| Use log-variance loss instead of KL loss (ablation) | PIS and DIS · DW (d=50, m=5, δ=2) | W²γ: PIS: 6.821 -> 6.823; DIS: 6.854 -> 6.855 | Hurt |
Surprising findings
- The general bridge objective with two controls remains numerically unstable and inefficient, so the authors focus on DIS and PIS for harder problems.
- Without tuning hyperparameters for the log-variance loss, it still substantially outperforms the KL loss in most settings.
Limitations
- General bridge framework with two controls is unstable and non-unique, and high-dimensional divergence computation is costly
- Results rely on hyperparameters not tuned for the log-variance loss, and the KL loss is sometimes slightly better
- Experiments are limited to a few synthetic benchmarks
Experimental setup
Benchmarks: GMM (d=2) Funnel (d=10) Double well (d=5, m=5, delta=4) Double well (d=50, m=5, delta=2)
Metrics: Delta log Z error Sinkhorn distance W2 normalized effective sample size Delta std of marginals
Baselines: PIS with reverse KL loss DIS with reverse KL loss DDS AIS SMC
Builds on: Path Integral Sampler Denoising Diffusion Sampler Time-Reversed Diffusion Sampler Schrödinger bridge log-variance loss for variational inference
Closest papers by meaning
- Transport meets Variational Inference: Controlled Monte Carlo Diffusions (ICLR 2024 · similarity 0.90)
- Rethinking Losses for Diffusion Bridge Samplers (NeurIPS 2025 · similarity 0.90)
- Variational Schrödinger Diffusion Models (ICML 2024 · similarity 0.90)
- Underdamped Diffusion Bridges with Applications to Sampling (ICLR 2025 · similarity 0.89)
- Learning non-equilibrium diffusions with Schrödinger bridges: from exactly solvable to simulation-free (NeurIPS 2025 · similarity 0.89)
- Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.