Research map / Optimization / Nonconvex and smooth optimization
Improved Particle Approximation Error for Mean Field Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that the finite-particle approximation error of mean-field Langevin dynamics (for entropy-regularized mean-field neural networks) is O(1/N) with no dependence on the LSI constant, and derives improved convergence, sampling and uniform-in-time Wasserstein propagation-of-chaos guarantees from it.
Problem
Finite-particle approximation of mean-field Langevin dynamics (MFLD) for entropy-regularized mean-field two-layer neural networks has error bounds that scale as 1/alpha (inverse LSI constant), which can deteriorate exponentially as regularization goes to zero.
Approach
Bounds the gap between the N-particle Gibbs objective and the mean-field objective using the Bregman divergence of the convex risk functional, relating it to the variance of an N-particle neural network average; this yields an LSI-constant-free O(1/N) error LR^2/(2N) instead of O(lambda/(alpha N)).
Key results
- Finite-particle objective gap bounded by LR^2/(2N) with no LSI-constant dependence, versus prior O(lambda/(alpha N)).
- Particle complexity for eps-accurate optimization improves from O(lambda/(alpha eps)) to O(1/eps).
- Uniform-in-time Wasserstein propagation of chaos error improves from O(1/(alpha^2 N)) to O(1/(alpha lambda N)).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the LSI-dependent particle approximation bound with the risk-structure-based bound | Mean-field neural networks · Particle approximation | objective gap: O(\frac{\lambda}{\alpha N}) -> LR^2/(2N) | Helped |
| Use the LSI-free particle approximation bound instead of the prior LSI-dependent bound | Finite-particle MFLD · MFLD optimization | particle complexity: N=O(\frac{\lambda}{\alpha \epsilon}) -> N = O(\frac{1}{\epsilon}) | Helped |
| Use the improved particle approximation bound in the Wasserstein propagation-of-chaos analysis | Finite-particle MFLD · Uniform-in-time Wasserstein propagation of chaos | particle approximation error: O(\frac{1}{\alpha^2N}) -> O(\frac{1}{\alpha\lambda N}) | Helped |
| Use the finite-particle MFLD bound for the marginal distribution | Finite-particle MFLD · Sampling guarantee for μ* | objective gap: \frac{LR^2}{2N} | Helped |
| Use a linear loss, for which the smoothness constant is zero | Mean-field neural networks · Particle approximation | objective gap: L=0 | Helped |
Limitations
- Iteration complexity still depends exponentially on the LSI constant, so polynomial complexity is not achieved.
- Requires uniform LSI assumption on the proximal Gibbs distribution and bounded-gradient / Lipschitz assumptions.
Experimental setup
Metrics: objective gap KL divergence Wasserstein propagation of chaos
Baselines: Chen et al. 2023 MFLD particle approximation bound O(lambda/(alpha N)) Monmarché 2024 uniform-in-time propagation of chaos Kook et al. 2024 discrete-time MFLD
Builds on: mean-field Langevin dynamics propagation of chaos one-step interpolation argument for Langevin dynamics Holley-Stroock perturbation
Closest papers by meaning
- Propagation of Chaos for Mean-Field Langevin Dynamics and its Application to Model Ensemble (ICML 2025 · similarity 0.94)
- Mean-field Underdamped Langevin Dynamics and its Spacetime Discretization (ICML 2024 · similarity 0.89)
- Mean-Field Langevin Dynamics for Signed Measures via a Bilevel Approach (NeurIPS 2024 · similarity 0.89)
- Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems (ICLR 2024 · similarity 0.89)
- Universal Approximation of Mean-Field Models via Transformers (ICML 2025 · similarity 0.87)
- Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.