Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Solving Differential Equations with Constrained Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes science-constrained learning (SCL), casting PDE solving as a constrained learning problem with worst-case (distributionally robust) losses, solved by a primal-dual algorithm with MH sampling of collocation points.
Problem
NN-based PDE solvers (PINNs, neural operators) are highly sensitive to collocation points and loss weights, often yielding low-quality or trivial solutions; penalty-based aggregation of expected losses does not target the weak solution of a PDE.
Approach
Shows that a (weak) PDE solution is equivalent to a constrained learning problem with worst-case losses; approximates the worst-case losses via truncated distributions psi_alpha and solves the resulting constrained problem with a primal-dual (gradient descent-ascent) method, also integrating invariance constraints and observations.
Key results
- On convection with beta=50, SCL achieves 3.74 ± 0.87% relative L2 error versus 56.5 ± 20% for PINN and 29.0 ± 33% for R3.
- On the eikonal problem, SCL reaches 9.95 ± 1.9% error versus 87.1% (PINN) and 85.5% (R3).
- For parametric convection, the PINN baseline needs about 6x more PDE-operator evaluations per epoch to match SCL's accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SCL instead of R3 | MLP solver · Convection β = 30 | relative L2 error: 0.999 ± 0.53 % -> 0.971 ± 0.30 % | Helped |
| Use SCL instead of R3 | MLP solver · Convection β = 50 | relative L2 error: 29.0 ± 33 % -> 3.74 ± 0.87 % | Helped |
| Use SCL instead of R3 | MLP solver · Reaction-diffusion (ν, ρ) = (3, 3) | relative L2 error: 0.736 ± 0.068 % -> 1.82 ± 0.74 % | Hurt |
| Use SCL instead of R3 | MLP solver · Reaction-diffusion (ν, ρ) = (3, 5) | relative L2 error: 0.665 ± 0.046 % -> 0.762 ± 0.11 % | Hurt |
| Use SCL instead of R3 | MLP solver · Eikonal | relative L2 error: 85.5 ± 27 % -> 9.95 ± 1.9 % | Helped |
| Add a period π/15 invariance constraint (ablation) | SCL for convection with β = 30 and fixed collocation points · Convection β = 30 | solution quality | Helped |
Surprising findings
- Enforcing only the observational constraint via a worst-case (maximum error) formulation for FNOs gives better prediction quality than minimizing average error.
- Fixed collocation points cause the constrained worst-case formulation to fail on convection, although the stochastic version succeeds.
Limitations
- Worst-case losses for infinite-dimensional parameters (e.g., forcing functions) are not yet supported; a fixed distribution is used.
- Single-BVP solving is about 5x more expensive than PINN.
- Sampling uses MH rather than faster gradient-based samplers, and convergence of gradient descent-ascent in non-convex settings is not fully understood.
Experimental setup
Datasets: convection, reaction-diffusion, eikonal, Helmholtz, Burgers', diffusion-sorption, Navier-Stokes BVPs (generated collocat
Benchmarks: convection BVP reaction-diffusion BVP eikonal BVP parametric convection Burgers' equation Navier-Stokes equation
Metrics: relative L2 error number of differential operator evaluations runtime
Baselines: PINN R3 FEM Fourier neural operator with average loss
Builds on: PINN Fourier neural operator constrained learning duality Metropolis-Hastings
Closest papers by meaning
- Learning differentiable solvers for systems with hard constraints (ICLR 2023 · similarity 0.88)
- Learning a Neural Solver for Parametric PDEs to Enhance Physics-Informed Methods (ICLR 2025 · similarity 0.88)
- Learning to Solve Differential Equation Constrained Optimization Problems (ICLR 2025 · similarity 0.87)
- Neural Spectral Methods: Self-supervised learning in the spectral domain (ICLR 2024 · similarity 0.87)
- Scaling physics-informed hard constraints with mixture-of-experts (ICLR 2024 · similarity 0.86)
- Graph Neural PDE Solvers with Conservation and Similarity-Equivariance (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.