Research map / Image generation / Fast diffusion sampling
Progressive Tempering Sampler with Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PTSD, which trains diffusion-based neural samplers sequentially across decreasing temperatures, using a finite-difference "temperature guidance" to extrapolate higher-temperature models to lower ones, refined by truncated importance resampling and local parallel tempering.
Problem
Neural samplers that amortize sampling from unnormalized densities are far less efficient in target evaluations than parallel tempering (PT), while PT yields only correlated samples and must be rerun whenever new samples are needed.
Approach
Train diffusion models sequentially across temperatures; extrapolate the score to a lower temperature via finite-difference temperature guidance (a contrast of the two highest-temperature models), then refine the generated buffer with truncated importance resampling and local PT steps before fine-tuning the next model.
Key results
- On MW-32, PTSD reaches W2 4.99 and TVD 0.14, with about 5.3e6 target evaluations versus roughly 1e9 to 2e10 for other neural samplers (DDS, CMCD, iDEM, BNEM, FAB, DiKL).
- On LJ-55, PTSD achieves W2 1.81 using about 1.5e5 target evaluations, compared with 1.3e10 for iDEM and 6.4e9 for BNEM; BNEM remains best on LJ-55 sample quality (W2 1.76).
- Ablation on MW-32 shows removing temperature guidance raises W2 from 4.99 to 24.59, and removing truncated IS raises it to 5.84.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PTSD instead of PT+DM | neural sampling on MoG-40 · MoG-40 | target density calls: 1.2\times 10^6 -> 1.0\times10^6 | Helped |
| Use PTSD instead of PT+DM | neural sampling on MW-32 · MW-32 | target density calls: 8.0\times10^6 -> 5.3\times10^6 | Helped |
| Use PTSD instead of PT+DM | neural sampling on LJ-55 · LJ-55 | target density calls: 1.65\times 10^5 -> 1.5\times 10^5 | Helped |
| Use PTSD instead of PT+DM | neural sampling on MoG-40 · MoG-40 | W2: 3.01 -> 1.93 | Helped |
| Use PTSD instead of PT+DM | neural sampling on MW-32 · MW-32 | W2: 5.04 -> 4.99 | Helped |
| Use PTSD instead of PT+DM | neural sampling on LJ-55 · LJ-55 | W2: 1.82 -> 1.81 | Hurt |
| Add temperature guidance (ablation) | PTSD · MW-32 | TVD: 0.34 -> 0.14 | Helped |
| Add temperature guidance (ablation) | PTSD · MW-32 | W2: 24.59 -> 4.99 | Helped |
| Add importance resampling (ablation) | PTSD · MW-32 | TVD: 0.23 -> 0.14 | Helped |
| Add importance resampling (ablation) | PTSD · MW-32 | W2: 5.84 -> 4.99 | Helped |
| Use PTSD instead of PT+DM | alanine dipeptide sampling · Alanine dipeptide | mean log-likelihood: 212.38 ± 0.05 -> 213.32 ± 0.06 | Helped |
| Use PTSD instead of PT+DM | alanine dipeptide sampling · Alanine dipeptide | KLD: 3.2e-02 -> 6.9e-2 | Hurt |
Surprising findings
- On alanine dipeptide, PT+DM achieves a lower Ramachandran KL divergence (3.2e-2) than PTSD (6.9e-2), even though PTSD has higher mean log-likelihood.
- PTSD is slower than PT in wall-clock time in the reported experiments, despite lower target evaluation counts.
Limitations
- Remains slower than parallel tempering in wall-clock time because a diffusion model must be fine-tuned at each temperature level.
- Progressive temperature decrease is sequential and not parallelizable across devices, unlike PT.
- Performance is sensitive to the temperature schedule, network learning quality, and the IS truncation threshold.
Experimental setup
Datasets: MoG-40 (synthetic target) Many-Well-32 (synthetic target) LJ-55 (Lennard-Jones potential) Alanine dipeptide (implicit solvent, Cartesian)
Benchmarks: MoG-40 Many-Well-32 LJ-55 Alanine dipeptide
Metrics: TVD of energy histograms MMD of energy histograms W2 distance number of target density evaluations mean absolute error of observable log-likelihood KL divergence of Ramachandran histograms
Baselines: FAB iDEM BNEM DiKL DDS CMCD PT+DM
Builds on: parallel tempering classifier-free guidance denoising diffusion samplers iDEM sequential Monte Carlo
Closest papers by meaning
- Progressive Inference-Time Annealing of Diffusion Models for Sampling from Boltzmann Densities (NeurIPS 2025 · similarity 0.87)
- Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.87)
- On scalable and efficient training of diffusion samplers (NeurIPS 2025 · similarity 0.87)
- Diffusion Path Samplers via Sequential Monte Carlo (NeurIPS 2026 · similarity 0.87)
- Learned Reference-based Diffusion Sampler for multi-modal distributions (ICLR 2025 · similarity 0.87)
- Sequential Controlled Langevin Diffusions (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.