Research map / Optimization / Implicit bias and generalization

Uniform-in-Time Weak Propagation of Chaos in Shallow Neural Networks

NeurIPS 2026 · Optimization › Implicit bias and generalization · approach: Gradient flow and mean-field dynamics · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proves non-asymptotic weak propagation of chaos for two-layer networks under gradient flow that holds uniformly in time, using the mean-field loss decay rate rather than strong convexity or log-Sobolev assumptions.

Problem

Finite-width gradient descent on two-layer networks in the feature-learning regime is only controlled by Grönwall bounds that grow exponentially in time, so existing propagation-of-chaos guarantees do not extend to long training horizons needed for small excess loss.

Approach

Bounds the finite-neuron function error uniformly in time by exploiting the decay of the mean-field excess loss (integrable square root of loss, or polynomial decay) together with a refined linearization that splits the source term into a constant near-convergence part and an O(1/m) higher-order part, instead of adding noise or assuming local strong convexity.</alpha_placeholder>

Key results

• Under L_t <= t^{-c} with c>=6, the uniform-in-time weak PoC error is O(d^2 log^2(dm)/m) for two-layer networks (corollary).
• For 2<c<6 the uniform bound scales as m^{-c/6}, and for c=2 as m^{-1/(6C)} up to constants.

Surprising findings

• Uniform-in-time weak PoC holds even though the strong form (Wasserstein) can fail when the Hessian spectrum decays to zero.

Limitations

• Requires loss decay rate assumption; guarantee is slower than 1/m for rates between t^{-2} and t^{-6}.
• Stated for one-hidden-layer networks with noiseless setting and regularity assumptions on the activation.

Experimental setup

Baselines: Grönwall estimate mean-field Langevin propagation of chaos

Builds on: mean-field analysis of two-layer networks Wasserstein gradient flow Grönwall-based propagation of chaos

Closest papers by meaning

• Uniform-in-time propagation of chaos for the mean-field gradient Langevin dynamics (ICLR 2023 · similarity 0.92)
• Quantitative Propagation of Chaos for SGD in Wide Neural Networks (NeurIPS 2020 · similarity 0.90)
• A Dynamical Central Limit Theorem for Shallow Neural Networks (NeurIPS 2020 · similarity 0.90)
• SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structures (NeurIPS 2025 · similarity 0.87)
• The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations (ICML 2025 · similarity 0.86)
• Flat Channels to Infinity in Neural Loss Landscapes (NeurIPS 2025 · similarity 0.85)