Research map / Probabilistic and scientific modeling / Differential equations and PDEs

Learning from Integral Losses in Physics Informed Neural Networks

ICML 2024 · Probabilistic and scientific modeling › Differential equations and PDEs · approach: Physics-informed neural network training · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that replacing integrals in PINN losses with unbiased single-sample estimates induces a parameter-dependent variance bias, and proposes deterministic sampling, double sampling, and a regularized delayed target method (N=1) that matches large-N integral estimates on Poisson, Maxwell-Ampere, and Smoluchowski problems.

Problem

Training physics-informed neural networks on partial integro-differential equations, where integrals or large sums in the PDE residual make accurate per-point evaluation impractical and naive unbiased sampling yields biased losses.

Approach

Shows the naive sampled MSE loss equals the true loss plus an excess variance term that depends on theta, then reduces that bias by using a detached, Polyak-averaged target network for the integral term (delayed target), with an added regularizer tying main and target outputs.

Key results

• Delayed target with N=1 achieves solution quality comparable to or slightly better than standard training with N=100 on Poisson, Maxwell, and Smoluchowski problems.
• Naive N=1 standard training yields overly smooth, biased solutions due to excess variance; double-sampling with N=1 diverges on the Poisson problem.
• Regularized delayed target (lambda=1) stabilizes training with M=1000 where unregularized M=100 diverges.

Measured effects reported in the paper

Surprising findings

• The double-sampling trick, which gives an unbiased full gradient, performed worse than the biased delayed target method in practice.
• Incomplete-gradient (TD-style) updates were preferable to full-gradient updates, consistent with off-policy RL findings.

Limitations

• Convergence guarantee holds only under linear function approximation and a discretized domain; nonlinear networks may not converge to reasonable solutions.
• Hyperparameters such as M, lambda, and batch size require manual setting; adaptive schemes left to future work.
• Experiments are on relatively small MLPs and synthetic PDE problems.

Experimental setup

Datasets: synthetic Poisson problems with Dirac-delta charges Maxwell-Ampere circuit problem Smoluchowski coagulation problem

Benchmarks: 2D Poisson with singular charges higher-dimensional Poisson (up to 10D) Maxwell-Ampere wired circuit Smoluchowski coagulation (2-4D)

Metrics: mean squared error to analytical solution normalized MSE

Baselines: standard MSE training with N=1 standard MSE training with N=100 deterministic sampling double-sampling trick Gaussian quadrature Leja quadrature quasi Monte Carlo

Builds on: physics-informed neural networks temporal difference learning double sampling Adam

Closest papers by meaning

• Number Theoretic Accelerated Learning of Physics-Informed Neural Networks (AAAI 2025 · similarity 0.88)
• Learning a Neural Solver for Parametric PDEs to Enhance Physics-Informed Methods (ICLR 2025 · similarity 0.87)
• Efficient Error Certification for Physics-Informed Neural Networks (ICML 2024 · similarity 0.87)
• Parameterized Physics-informed Neural Networks for Parameterized PDEs (ICML 2024 · similarity 0.87)
• Dual Cone Gradient Descent for Training Physics-Informed Neural Networks (NeurIPS 2024 · similarity 0.87)
• Sub-Sequential Physics-Informed Learning with State Space Model (ICML 2025 · similarity 0.87)