Research map / Learning theory and algorithms / Kernels and regression theory
Mean-field Underdamped Langevin Dynamics and its Spacetime Discretization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes N-particle underdamped Langevin algorithm (NULA), a spacetime discretization of mean-field underdamped Langevin dynamics with exponential-integrator steps, giving faster TV convergence guarantees for entropy-regularized mean-field optimization.
Problem
Optimizing entropy-regularized non-linear functionals over probability measures (e.g., mean-field neural networks, MMD minimization, KSD minimization) with a practical, implementable discretization of mean-field underdamped Langevin dynamics and non-asymptotic guarantees in total variation.
Approach
Uses an exponential-integrator time discretization of mean-field underdamped Langevin dynamics, combined with a particle approximation of the mean-field interaction, and a refined hypocoercivity Lyapunov analysis with damping gamma = sqrt(L) to get faster mixing and smaller particle-number bias than prior spacetime discretizations of mean-field Langevin dynamics.
Key results
- Theory: NULA with gamma=sqrt(L) reaches TV error epsilon with step size h=Õ(C_LSI epsilon/(L^{3/2} d^{1/2})), iterations K=Õ(κ² d^{1/2}/ε), and particles N=Θ(κ² d/ε²), improving on prior spacetime results (Õ(κ² L d/ε²) mixing).
- Experiments on a two-layer tanh mean-field network (d=1000, n=100) show N-ULA achieves lower loss than N-LA and EM-N-ULA for N=512, 1024, 2048, and avoids the convergence instability seen at N=256 for N-ULA.
- The mean-field underdamped dynamics convergence rate improves from O(C_LSI/L) to O(C_LSI/sqrt(L)) by choosing gamma=sqrt(L).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MULA instead of MLA | Mean-field Langevin optimization · TV-distance mixing time | mixing time: \widetilde{\Theta}\Big(\kappa^2 \mathscr{L}d/ \epsilon^2\Big) -> \widetilde{\Theta}\left(\kappa^{2}d^{1/2}/\epsilon\righ | Helped |
| Use the proposed N-particle underdamped system with γ=√L | N-particle underdamped Langevin dynamics · TV-distance space discretization | mixing time: \widetilde{\Theta}\Big(\kappa\Big) -> {\widetilde{\Theta}\Big(\kappa/\mathscr{L}^{1/2}\Big)} | Helped |
| Use N-ULA instead of the prior N-particle Langevin algorithm | N-particle spacetime discretization · TV-distance spacetime discretization | mixing time: \widetilde{\Theta}\left(\kappa^2\mathscr{L}d/\epsilon^2\right) -> \widetilde{\Theta}\Big(\kappa^2d^{1/2}/\epsilon\Big) | Helped |
| Use N-ULA instead of the prior N-particle Langevin algorithm | N-particle spacetime discretization · TV-distance spacetime discretization | number of particles: ${\Theta}\left(\kappa\mathscr{L}^3d/\epsilon^2\right) -> {$\Theta\left(\kappa^2d/\epsilon^2\right)$} | Helped |
| Use N-ULA rather than N-LA | Mean-field neural-network training · Gaussian-function mean-field neural-network experiment | training loss | Helped |
| Use exponential-integrator N-ULA rather than EM-N-ULA | Mean-field neural-network training · Gaussian-function mean-field neural-network experiment | training loss | Helped |
| Use N-ULA at N=256 | Mean-field neural-network training · Gaussian-function mean-field neural-network experiment, N=256 | training loss | Mixed |
| Use N-ULA rather than N-LA or EM-N-ULA | Mean-field neural-network training · Gaussian-function mean-field neural-network experiment, N=512, 1024, 2048 | training loss | Helped |
| Set damping coefficient γ=√L instead of γ=1 (ablation) | Mean-field underdamped Langevin dynamics · Continuous-time convergence | energy-gap decay rate: O(\mathscr{C}_{\textsf{LSI}}/\mathscr{L}) -> O(\mathscr{C}_{\textsf{LSI}}/\sqrt{\mathscr{L}}) | Helped |
| Replace the EM spacetime discretization with the exponential-integrator discretization (ablation) | N-particle underdamped Langevin algorithm · Gaussian-function mean-field neural-network experiment | discretization bias | Helped |
Surprising findings
- The EM discretization of the underdamped particle system incurs larger bias than the exponential integrator in experiments.
- N-ULA with N=256 exhibits convergence instability (loss rises after many epochs), while larger particle counts do not.
Limitations
- Requires additional assumptions (bounded gradient, second-moment bounds, initial functional bounds) beyond the convexity, smoothness and LSI assumptions of prior work.
- Dependence on the LSI constant is not improved relative to MLA-type methods; the analysis loses an extra C_LSI factor after Talagrand's inequality.
- Guarantees are in TV distance rather than energy gap or KL, so rates are not directly comparable to continuous-time energy convergence.
Experimental setup
Datasets: synthetic Gaussian function regression data
Metrics: empirical training loss versus epochs
Baselines: N-LA (spacetime discretization of mean-field Langevin dynamics) EM-N-ULA (Euler-Maruyama discretization of N-particle underdamped Langevin) MLA (mean-field Langevin algorithm)
Builds on: mean-field Langevin dynamics underdamped Langevin Monte Carlo hypocoercivity analysis of mean-field underdamped Langevin dynamics
Closest papers by meaning
- Improved Particle Approximation Error for Mean Field Neural Networks (NeurIPS 2024 · similarity 0.89)
- Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics (ICLR 2025 · similarity 0.88)
- Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems (ICLR 2024 · similarity 0.88)
- Propagation of Chaos for Mean-Field Langevin Dynamics and its Application to Model Ensemble (ICML 2025 · similarity 0.87)
- Mean-Field Langevin Dynamics for Signed Measures via a Bilevel Approach (NeurIPS 2024 · similarity 0.86)
- The Poisson Midpoint Method for Langevin Dynamics: Provably Efficient Discretization for Diffusion Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.