Research map / Optimization / Nonconvex and smooth optimization
Mean-Field Langevin Dynamics for Signed Measures via a Bilevel Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that mean-field Langevin dynamics can solve signed-measure convex problems via a bilevel reduction, which beats the lifting reduction, and gives an annealing schedule with improved rates plus local LSI bounds for single-neuron learning.
Problem
Convex optimization over finite signed measures with squared total-variation regularization (e.g., infinite-width two-layer networks, sparse deconvolution) is not directly amenable to mean-field Langevin dynamics, which is defined over probability measures.
Approach
Reduces signed-measure problems to probability measures via a bilevel variational representation of the squared TV norm, showing this (unlike the lifting reduction) satisfies displacement smoothness and uniform LSI; uses an adapted annealing schedule whose LSI constant is controlled along the trajectory, giving time complexity exp(O(λ^{-1} log λ^{-1})) instead of exp(O(λ^{-2})).
Key results
- Bilevel annealing schedule reaches (1+Δ)-multiplicative accuracy in time exp(O(λ^{-1} log λ^{-1})), versus exp(O(λ^{-2})) for the classical log(t) schedule.
- Lifting reduction with q_r=q_w=b=1 fails to satisfy local LSI for small λ, while the bilevel reduction satisfies both displacement smoothness and uniform LSI under mild assumptions.
- For single-neuron learning on the sphere, the local LSI constant at optimum is independent of β up to polynomial dependence on d and β (β ≥ poly(d, λ^{-1})) rather than exponential.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the bilevel reduction instead of the lifting reduction | MFLD for signed-measure optimization · Convex optimization over signed measures | convergence guarantees and rates | Helped |
| Use the proposed bilevel annealing schedule instead of the classical schedule | MFLD-Bilevel · Fixed multiplicative accuracy for signed-measure optimization | time complexity: exp(c λ^{-1} log λ^{-1}) instead of exp(c' λ^{-2}) | Helped |
| Apply a local Lyapunov-type analysis at the optimum | MFLD-Bilevel for learning a single neuron · Single-neuron learning | dependence of the local LSI constant on dimension and noise level | Helped |
Limitations
- Lifting reduction is incompatible with MFLD for small λ, so the analysis focuses on the bilevel approach.
- Local LSI analysis is restricted to single-neuron teacher, rotationally invariant data and smooth monotone activations.
- Discrete-time guarantees for MFLD on Riemannian manifolds are not studied.
Experimental setup
Metrics: objective suboptimality local LSI constant
Baselines: lifting reduction MFLD classical log(t) annealing schedule constant-temperature MFLD
Builds on: mean-field langevin dynamics multiple kernel learning variational representation Holley-Stroock perturbation
Closest papers by meaning
- Improved Particle Approximation Error for Mean Field Neural Networks (NeurIPS 2024 · similarity 0.89)
- Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics (ICLR 2025 · similarity 0.88)
- Propagation of Chaos for Mean-Field Langevin Dynamics and its Application to Model Ensemble (ICML 2025 · similarity 0.88)
- Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems (ICLR 2024 · similarity 0.88)
- Mean-field Underdamped Langevin Dynamics and its Spacetime Discretization (ICML 2024 · similarity 0.86)
- Convergence of Mean-Field Langevin Stochastic Descent-Ascent for Distributional Minimax Optimization (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.