Research map / Probabilistic and scientific modeling / Variational and generative sampling
Preconditioned Langevin Dynamics with Score-based Generative Models for Infinite-Dimensional Linear Bayesian Inverse Problems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes Langevin sampling with score-based generative priors directly in infinite-dimensional Hilbert space, deriving score-error-dependent KL bounds and an optimal preconditioner for uniform convergence across posterior modes.
Problem
Bayesian linear inverse problems in infinite-dimensional function spaces: sampling with score-based generative priors via Langevin dynamics is unstable and not well-defined under discretization refinement without understanding of score approximation error and preconditioning.
Approach
Formulates the SGM-driven preconditioned Langevin sampler directly in Hilbert space, derives KL error estimates depending explicitly on score approximation error, and shows existence and closed form of an optimal trace-class preconditioner that depends on the score error and the forward operator, giving uniform convergence across modes for Gaussian and non-Gaussian priors.
Key results
- Derives mode-by-mode and global KL error estimates for the preconditioned Langevin sampler in the Gaussian setting, with sufficient conditions for globally bounded error in infinite dimensions.
- Shows the optimal preconditioner at leading order is C = [C_mu^{-1} + sigma^{-2} A^T A]^{-1} (or C_mu when A is absent), with a first-order correction in tau that depends on the score error.
- Illustrations on Brownian sheet and inverse heat source problems show the optimal preconditioner is robust to score error while vanilla Langevin deteriorates as score error increases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the optimal preconditioner C = [C_mu^{-1} + sigma^{-2} A^T A]^{-1} instead of vanilla Langevin | Langevin sampling with score-based generative priors · Inverse heat source problem | convergence rate across modes | Helped |
| Use a trace-class preconditioner instead of the identity | Infinite-dimensional Langevin sampling · Infinite-dimensional linear inverse problems | stability | Helped |
| Use vanilla Langevin without preconditioning | Langevin sampling with score-based generative priors · Inverse heat source problem | reconstruction stability | Hurt |
| Match the preconditioner to the prior covariance and forward operator | Preconditioned Langevin dynamics · Gaussian linear inverse problems | convergence rate across posterior modes | Helped |
| Use a preconditioner close to the prior covariance | Preconditioned Langevin dynamics · Brownian sheet reconstruction | discretization invariance | Helped |
| Keep the score's multiplicative approximation error small relative to the preconditioner | Preconditioned Langevin dynamics · Gaussian and non-Gaussian linear inverse problems | stationary-distribution error | No clear effect |
| Add an additive score approximation error | Preconditioned Langevin dynamics · Gaussian and non-Gaussian linear inverse problems | posterior bias | Hurt |
| Use an approximate score with the optimal preconditioner | Preconditioned Langevin dynamics · Inverse heat source problem | robustness to score approximation error | Helped |
Surprising findings
- Vanilla Langevin with identity preconditioner produces samples with infinite energy, outside the Hilbert space, even though numerics look stable.
- The bias from the score error on the mean is amplified by inverse eigenvalues and cannot be removed by preconditioning.
Limitations
- Assumes finite-dimensional data, co-diagonalizability of prior and noise covariance, and typically diagonal A^T A.
- Restricted to linear inverse problems; nonlinear case left open.
- Score error modeled through simplified mode-wise assumption rather than general neural network error.
Experimental setup
Metrics: KL divergence mode autocorrelation
Baselines: vanilla Langevin sampler
Builds on: Langevin dynamics score-based diffusion models in Hilbert spaces preconditioned Crank-Nicolson
Closest papers by meaning
- The Expected Loss of Preconditioned Langevin Dynamics Reveals the Hessian Rank (AAAI 2024 · similarity 0.90)
- Dimension-Uniform Discretization Analysis of Preconditioned Annealed Langevin Dynamics for Multimodal Gaussian Mixtures (NeurIPS 2026 · similarity 0.89)
- Solving Linear-Gaussian Bayesian Inverse Problems with Decoupled Diffusion Sequential Monte Carlo (ICML 2025 · similarity 0.89)
- Consistency Posterior Sampling for Diverse Image Synthesis (CVPR 2025 · similarity 0.88)
- Wasserstein Convergence of Critically Damped Langevin Diffusions (NeurIPS 2025 · similarity 0.88)
- Posterior Sampling by Combining Diffusion Models with Annealed Langevin Dynamics (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.