Research map / Learning theory and algorithms / Combinatorial and search optimization
Physics-Informed Neural Network Policy Iteration: Algorithms, Convergence, and Verification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ELM-PI and PINN-PI, neural policy iteration algorithms that solve the linear GHJB equations with extreme-learning-machine least squares or physics-informed networks, with convergence analysis to viscosity solutions and dReal-based stability verification.
Problem
Solving nonlinear optimal control problems (HJB equations) in high dimensions, where policy iteration requires solving linear GHJB PDEs that Galerkin methods handle poorly, and where convergence to viscosity solutions and closed-loop stability are not guaranteed.
Approach
Solve each policy-evaluation GHJB with neural approximators: ELM-PI reduces each step to linear least squares over random-feature networks, while PINN-PI trains a neural network with a PDE residual loss plus a Frobenius-norm loss on the linearized feedback gain to encourage local stability; convergence is proven to viscosity solutions and stability is checked with SMT (dReal) verification.
Key results
- On a synthetic n-dimensional nonlinear control problem, ELM-PI is more accurate and faster than PINN-PI for n≤3, while PINN-PI reaches about 1e-2 error with m=800 across all tested dimensions.
- For the inverted pendulum, a controller from ELM-PI with m=50 was found to be non-stabilizing, while m=100 was verified stabilizing with dReal.
- PINN-PI converges to equilibrium in under two seconds on cartpole and quadrotor benchmarks where PPO, HJBPPO and CT-MBRL costs diverge.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ELM-PI rather than PINN-PI | policy iteration · Synthetic problems with n≤3 | computational efficiency and approximation accuracy | Helped |
| use PINN-PI rather than ELM-PI | policy iteration · Synthetic problems with n≥5 | computational efficiency | Helped |
| increase ELM-PI hidden units to m=3200 or m=6400 (ablation) | ELM-PI · Synthetic n=4 nonlinear control | maximum testing error: 10^{-3}; 10^{-5} | Helped |
| increase PINN-PI hidden units, collocation points, or training steps (ablation) | PINN-PI · Synthetic n-dimensional nonlinear control | maximum testing error | No clear effect |
| use ELM-PI with m=100 and N=200 rather than m=50 and N=100 (ablation) | ELM-PI · Inverted pendulum | stability | Helped |
| add the local-stability loss term to PINN-PI (ablation) | PINN-PI · High-dimensional control problems | stability | Helped |
| remove the local-stability loss term (ablation) | PINN-PI · High-dimensional control problems | stability | Hurt |
| use PINN-PI rather than PPO, HJBPPO, or CT-MBRL | control policy · Cartpole and quadrotor | stability | Helped |
| use PINN-PI rather than PPO, HJBPPO, or CT-MBRL | control policy · Inverted pendulum | control cost | No clear effect |
| use ELM-PI rather than successive Galerkin approximations | GHJB solving · Low-dimensional control problems | solver time | Helped |
Surprising findings
- A value function and controller that appear converged over policy iterations can still be unstable (m=50 ELM-PI case), motivating formal verification.
- Standard RL baselines (PPO, HJBPPO, CT-MBRL) fail to stabilize cartpole and quadrotor, with accumulated control cost diverging.
Limitations
- ELM-PI becomes inefficient for dimensions n≥5 due to the required number of hidden units.
- PINN-PI accuracy does not improve significantly as m, N, or training steps increase.
- Convergence of PINN-PI relies on an unverified Lipschitz-constant hypothesis and cannot guarantee global minima of the nonconvex loss.
Experimental setup
Metrics: maximum testing error relative to true value function computational time control cost
Baselines: successive Galerkin approximations PPO HJBPPO CT-MBRL
Builds on: policy iteration physics-informed neural networks extreme learning machine successive Galerkin approximations Kleinman's algorithm
Closest papers by meaning
- RoPINN: Region Optimized Physics-Informed Neural Networks (NeurIPS 2024 · similarity 0.89)
- Sub-Sequential Physics-Informed Learning with State Space Model (ICML 2025 · similarity 0.89)
- Physics-informed Neural Networks for Functional Differential Equations: Cylindrical Approximation and Its Convergence Guarantees (NeurIPS 2024 · similarity 0.88)
- Physics-Informed Representation and Learning: Control and Risk Quantification (AAAI 2024 · similarity 0.88)
- Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective (NeurIPS 2025 · similarity 0.88)
- Efficient Error Certification for Physics-Informed Neural Networks (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.