Research map / Optimization / Implicit bias and generalization

Quantitative convergence of trained neural networks to Gaussian processes

NeurIPS 2025 · Optimization › Implicit bias and generalization · approach: Gradient flow and mean-field dynamics · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Gives finite-width quantitative bounds (Wasserstein-1 via Stein's method) on how quantum neural networks at initialization and under gradient-flow training approach their Gaussian process limits, uniform in training time.

Problem

Qualitative convergence of quantum neural networks (observable-sum output) to Gaussian processes in the infinite-width limit is known, but finite-width quantitative rates, especially during training, are lacking.

Approach

Applies multidimensional Stein's method to bound the Wasserstein-1 distance at initialization using light-cone sizes and a normalization constant N(m) tied to barren plateaus; then bounds parameter drift under gradient flow to get lazy-regime control and a truncated W1 bound uniform in time.

Key results

• At initialization, the Wasserstein-1 distance to N(0, K_0) is bounded by N^{3/2}(2||K_0^{-1}||·||K_0||^{1/2}+6)·m|M|^{7/2}|N|^{7/2}/N(m)^3·(1+log N(m)), vanishing as m grows if barren plateaus are absent.
• For trained networks under gradient flow, the truncated W1 distance to the Gaussian process is bounded by a constant times L^2 m^{9/4}|M|^{11/2}|N|^{9/2}/N(m)^5·(1+log N(m)), uniform in training time t.
• Lazy training bound: sup over inputs and time of |f(Θ_t,x) - f^lin| is bounded by 132 n^2 R^2(δ)(1+λ^{-3})·L^2 m^2 |M|^5 |N|^2 log N(m)/N(m)^5 with probability at least 1-δ.

Surprising findings

• The bounds scale as N̄^{3/2} in the number of possible inputs, worse than the classical central-limit-theorem rate for independent sums.

Limitations

• Bounds depend on the normalization N(m), which decays with depth due to barren plateaus, so they are only useful when barren plateaus are absent.
• Scaling in the number of inputs N̄ is polynomial and worse than classical CLT bounds.
• Requires assumptions on the minimal eigenvalue of the NTK and on the light-cone sizes.

Experimental setup

Metrics: Wasserstein distance of order 1 truncated Wasserstein distance of order 1

Builds on: Girardi et al. quantum neural network Gaussian process convergence Stein's method for multivariate normal approximation neural tangent kernel

Closest papers by meaning

• Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent (NeurIPS 2019 · similarity 0.90)
• A Dynamical Central Limit Theorem for Shallow Neural Networks (NeurIPS 2020 · similarity 0.89)
• Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit (NeurIPS 2026 · similarity 0.88)
• An Infinite-Width Analysis on the Jacobian-Regularised Training of a Neural Network (ICML 2024 · similarity 0.88)
• Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $\mu$ Parametrization (ICML 2025 · similarity 0.87)
• On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling (NeurIPS 2025 · similarity 0.86)