Research map / Optimization / Implicit bias and generalization

Global convergence of adjoint-optimized neural PDEs

NeurIPS 2026 · Optimization › Implicit bias and generalization · approach: Neural tangent kernel analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proves global convergence of adjoint gradient descent for training neural-network source terms in nonlinear parabolic PDEs in the joint infinite-width and infinite-training-time limit, showing the NN-PDE solution converges weakly to the target data.

Problem

Calibrating neural-network terms embedded in nonlinear parabolic PDEs via adjoint gradient descent yields a non-convex problem even in the infinite-width limit; no global convergence guarantees existed for nonlinear NN-PDE training.

Approach

Shows that in the infinite-width limit the NN source term follows a constant limit NTK dynamics; the quadratic adjoint functional Q_tau decays to zero via a cycle-of-stopping-times argument with a second-level adjoint regularity bound, so the PDE solution converges weakly to the target h as training time goes to infinity.

Key results

• Proves that as the number of hidden units N tends to infinity the trained NN-PDE solution converges to the infinite-width limit system uniformly over finite training horizons.
• Proves the loss J(tau) is monotonically decreasing with dJ/dtau = -alpha_tau Q_tau and that the NN-PDE solution converges weakly to the target h in L2 as training time tends to infinity.
• Shows that strong limit points of the trained NN-PDE solution are global minimizers of the loss.

Surprising findings

• The limit training problem is non-convex in the NN function even in the infinite-width limit, unlike standard infinite-width NN training.

Limitations

• Requires strong assumptions (e.g., bounded sigma, Lipschitz q, compactly supported output-weight initialization, Robbins-Monro plus alpha^{4/3} integrability learning rate).
• Results are asymptotic (N to infinity, tau to infinity) and do not give rates for finite width.
• Convergence is weak in L2 for the solution; strong convergence only for limit points.

Experimental setup

Builds on: neural tangent kernel adjoint method for PDE optimization physics-informed neural networks

Closest papers by meaning

• How does PDE order affect the convergence of PINNs? (NeurIPS 2024 · similarity 0.88)
• Graph Neural PDE Solvers with Conservation and Similarity-Equivariance (ICML 2024 · similarity 0.88)
• Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioning (NeurIPS 2024 · similarity 0.87)
• Active Learning for Neural PDE Solvers (ICLR 2025 · similarity 0.87)
• Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification (ICML 2024 · similarity 0.87)
• Global Convergence in Neural ODEs: Impact of Activation Functions (ICLR 2025 · similarity 0.87)