Research map / Optimization / Nonconvex and smooth optimization
Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Provides the first KL convergence analysis of geometric-tempered Langevin dynamics under functional inequalities (continuous and discrete time), derives optimal schedules for strongly log-concave cases, and gives lower bounds showing geometric tempering can be exponentially slow or worsen log-Sobolev/Poincaré constants.
Problem
Theoretical understanding of when geometric tempering helps Langevin dynamics sampling multi-modal distributions, and how to choose the temperature schedule.
Approach
Upper bounds on KL convergence of tempered Langevin in terms of inverse log-Sobolev constants of the geometric path and the schedule's derivative, plus matching-style lower bounds (TV) showing exponential slowdown for bimodal and well-conditioned targets.
Key results
- Continuous-time KL upper bound for tempered Langevin, with optimal schedule lambda(s)=min(alpha_nu/(alpha_nu-alpha_pi)*(1+alpha_nu s)/(2+alpha_nu s),1) when alpha_pi<alpha_nu, and vanilla Langevin optimal when alpha_pi >= alpha_nu/2.
- Geometric tempering can make Poincare constant exponentially worse: C_P(mu_lambda) >= exp(m^2(1-lambda)/100)/(4e4 m) - m^2 while C_LS(nu)=1 and C_LS(pi)<=81m^5.
- For a bimodal mixture of Gaussians, TV(p^K, pi) >= 1/20 - 16 e^{-m^2/64} sum T_k, so exponential total time in m is required.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use geometric tempering between the Gaussian proposal and the specified target | Tempered Langevin dynamics · Geometric tempering, λ ∈ [1/2, 1] | Poincaré constant: C_P(μ_λ) ≥ 1/(4·10^4m) e^{m^2(1 - λ)/100} - m^2 | Hurt |
| Use a standard Gaussian proposal with geometric tempering | Tempered Langevin dynamics on the bimodal Gaussian-mixture target · Bimodal Gaussian-mixture target | TV distance: TV(p^K_{T_K}, π) ≥ 1/20 - 16·e^{-m^2/64} · ∑_{k = 1}^K T_k | Hurt |
| Use a standard Gaussian proposal with geometric tempering | Tempered Langevin dynamics on the well-conditioned target · Well-conditioned target | TV distance: TV(p^k_{T_k}, π) ≥ 1/5 - δ_k - 10m · √δ_k · ∑_{i = 1}^k T_i | Hurt |
| Use vanilla Langevin instead of tempering | Strongly log-concave proposal and target with α_π ≥ α_ν · Strongly log-concave setting | Upper-bound functional G_t(λ) | No clear effect |
| Use the optimal custom tempering schedule instead of vanilla Langevin | Strongly log-concave proposal and target with α_π < α_ν/2 · Strongly log-concave setting | Upper-bound functional G_t(λ) | Helped |
| Use a linear tempering schedule instead of vanilla Langevin | Strongly log-concave proposal and target with α_ν > α_π · Short versus large time horizons | Upper-bound functional G_t(λ): G_t(λ) ∼ 1/(2 α_π t) | Mixed |
| Use the optimal schedule instead of linear tempering or vanilla Langevin | Strongly log-concave example with α_π = 0.01 and α_ν = 1.0 · Different time horizons | Upper-bound functional G | Mixed |
Surprising findings
- Even for a well-conditioned uni-modal-like target with decent log-Sobolev constant, geometric tempering from a Gaussian proposal can exhibit exponentially slow convergence.
- Geometric tempering can exponentially worsen log-Sobolev and Poincare constants even when proposal and target have favorable functional inequalities.
Limitations
- Upper bounds are in terms of log-Sobolev constants of intermediate distributions that are hard to compute; closed-form optimal schedules only in strongly log-concave case.
- Lower bounds are for specific toy examples (Gaussian mixtures, Gaussian proposal) and do not cover general targets.
- Optimal schedule depends on unknown alpha_pi in practice.
Experimental setup
Metrics: KL divergence bound total variation distance lower bound log-Sobolev constant Poincare constant
Baselines: standard Langevin dynamics (lambda equiv 1) linear tempering schedule
Builds on: unadjusted Langevin algorithm annealed Langevin dynamics parallel tempering
Closest papers by meaning
- Forward-KL Convergence of Time-Inhomogeneous Langevin Diffusions (ICML 2026 · similarity 0.90)
- Langevin Monte Carlo for strongly log-concave distributions: Randomized midpoint revisited (ICLR 2024 · similarity 0.88)
- Restricted Spectral Gap Decomposition for Simulated Tempering Targeting Mixture Distributions (NeurIPS 2025 · similarity 0.87)
- The Poisson Midpoint Method for Langevin Dynamics: Provably Efficient Discretization for Diffusion Models (NeurIPS 2024 · similarity 0.87)
- A connection between Tempering and Entropic Mirror Descent (ICML 2024 · similarity 0.87)
- Constrained Sampling with Primal-Dual Langevin Monte Carlo (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.