Research map / Optimization / Implicit bias and generalization
How does PDE order affect the convergence of PINNs?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows via gradient-flow analysis of two-layer ReLU^p PINNs that convergence probability falls with PDE order and dimension, and that variable splitting into lower-order systems improves convergence.
Problem
Why PINNs struggle to optimize high-order and high-dimensional PDEs; the theoretical link between PDE order, dimension, ReLU power, and gradient-flow convergence.
Approach
Extends the two-layer PINN gradient-flow convergence analysis of Gao et al. to general kth-order linear PDEs and ReLU^p activations with tighter bounds, then proves that variable splitting reducing the derivative order to |ξ| gives a higher convergence probability and allows lower p.
Key results
- Theorem 3.2 gives a width requirement that grows exponentially with PDE order k, dimension d, and ReLU power p, with a polylogarithmic bound in 1/δ (tighter than the δ^-3 of Gao et al. for p=3).
- Variable splitting reduces the required power to p ≥ |ξ|+1 and the derivative order, and in experiments VS-PINNs converge more effectively than vanilla PINNs on the heat equation over five seeds.
- On the fourth-order elastic beam equation, the finest splitting with first-order derivatives (p=2) yields lower loss and lower variance than PINN with p=5 and than a partial split.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase PDE order from second to fourth | PINNs · Poisson and bi-harmonic equations | gradient-flow convergence | Hurt |
| decrease ReLU power p | PINNs · Second-order heat equation | training loss convergence | Helped |
| use variable splitting to reduce the derivative order in the loss | PINNs · Second-order heat equation | training loss convergence | Helped |
| use finest variable splitting (|ξ| = 1, p = 2) instead of coarser splitting (|ξ| = 2, p = 3) (ablation) | VS-PINNs · Fourth-order elastic beam equation | training loss reduction | Helped |
Surprising findings
- Larger ReLU power p can hurt convergence from an optimization perspective, despite better approximation capacity.
- Training PINNs for fourth-order PDEs with gradient descent did not proceed properly, requiring Adam for the experiments.
Limitations
- Only sufficient conditions for convergence are established; necessary conditions are open.
- Analysis is limited to empirical losses with fixed collocation points and continuous-time gradient flow rather than discrete gradient descent.
- Results cover two-layer MLPs and linear PDEs; convergence for Adam, L-BFGS and other activations remains open.
Experimental setup
Datasets: uniform grid collocation points for Poisson, bi-harmonic, heat, convection-diffusion, elastic beam equations
Benchmarks: Poisson equation bi-harmonic equation second-order heat equation convection-diffusion equation elastic beam equation
Metrics: training loss convergence
Baselines: PINN VS-PINN
Builds on: Gao et al. 2023 PINN gradient flow convergence PINNs variable splitting PINNs
Closest papers by meaning
- Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective (NeurIPS 2025 · similarity 0.88)
- Challenges in Training PINNs: A Loss Landscape Perspective (ICML 2024 · similarity 0.88)
- Global convergence of adjoint-optimized neural PDEs (NeurIPS 2026 · similarity 0.88)
- RoPINN: Region Optimized Physics-Informed Neural Networks (NeurIPS 2024 · similarity 0.88)
- PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs (NeurIPS 2024 · similarity 0.87)
- PINN Balls: Scaling Second-Order Methods for PINNs with Domain Decomposition and Adaptive Sampling (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.