Research map / Probabilistic and scientific modeling / Differential equations and PDEs

PINN Balls: Scaling Second-Order Methods for PINNs with Domain Decomposition and Adaptive Sampling

NeurIPS 2025 · Probabilistic and scientific modeling › Differential equations and PDEs · approach: Physics-informed neural network training · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PINN Balls, a sparse local mixture-of-experts of PINNs with RBF-based learnable domain decomposition trained by adversarial adaptive sampling and scalable Levenberg-Marquardt second-order updates.

Problem

Second-order training of PINNs scales poorly in memory with parameter count, and domain decomposition for PINNs is usually fixed a priori rather than learned.

Approach

Combine a local mixture of experts with compactly supported quartic RBF gating (so the Jacobian is sparse) and train the RBF centers/radii jointly with the network via adversarial adaptive sampling, enabling memory-feasible Levenberg-Marquardt updates; also provides convergence results for AAS with Dirichlet and entropy regularizers.

Key results

• On Helmholtz, PINN Balls (5 experts) reaches relative L2 error 2.8e-9 vs 1.2e-3 for XPINN and APINN; PINN Balls (20) reaches 1.3e-9.
• On Burgers, PINN Balls (5) reaches 2.3e-4 relative L2 error, comparable to APINN (9.1e-4) and far better than XPINN (1.3e-1); PINN Balls (20) reaches 1.7e-3.
• Dense LM solver runs out of memory at roughly 8000 parameters on the supervised test, while sparse PINN Balls handle much larger models; global LM on Navier-Stokes runs out of memory around 2000 parameters.

Measured effects reported in the paper

Surprising findings

• Larger ensembles do not necessarily outperform smaller ones at a fixed parameter count on Burgers (PINN Balls with 20 experts had higher error than with 5).
• The dense architecture is more memory efficient than the 5-expert model because sparse storage of indices inflates memory when Jacobian is about 30% nonzero.

Limitations

• Sparse Jacobian assembly and sparse solver overhead become non-negligible at moderate problem sizes; autodiff packages lack backprop through sparse matrices.
• Slow convergence if individual experts have too few parameters or are poorly initialized; DD training depends strongly on initialization.
• Theoretical convergence to uniform sampling assumes vanishing residuals, which practice does not satisfy; first-order DD optimizer is mismatched with second-order LM step.

Experimental setup

Datasets: Helmholtz PDE instance (k_x=4, k_y=1) Burgers equation instance with public reference solution 2D cylinder flow Navier-Stokes (Re=100) reference solution

Benchmarks: Helmholtz equation Burgers' equation Navier-Stokes flow past a cylinder

Metrics: relative L2 error training loss runtime Jacobian non-zero percentage memory consumption

Baselines: XPINN APINN FBPINN dense PINN with LM

Builds on: Adversarial Adaptive Sampling PINN XPINN FBPINN APINN

Closest papers by meaning

• When and Why Adversarial Training Improves PINNs: A Neural Tangent Kernel Perspective (NeurIPS 2026 · similarity 0.89)
• Challenges in Training PINNs: A Loss Landscape Perspective (ICML 2024 · similarity 0.88)
• Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEs (ICLR 2024 · similarity 0.88)
• PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks (ICLR 2024 · similarity 0.88)
• Efficient Error Certification for Physics-Informed Neural Networks (ICML 2024 · similarity 0.88)
• Sub-Sequential Physics-Informed Learning with State Space Model (ICML 2025 · similarity 0.88)