Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Scaling physics-informed hard constraints with mixture-of-experts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PI-HC-MoE, which enforces hard PDE constraints locally via mixture-of-experts over decomposed spatiotemporal domains, each solved by a differentiable least-squares expert using implicit differentiation, for neural PDE solvers.
Problem
Hard physics-informed constraints via differentiable optimization become computationally and memory expensive on large spatiotemporal meshes, and soft penalty constraints fail to enforce physics accurately.
Approach
Partition the spatiotemporal domain among K experts, each solving a local Levenberg-Marquardt least-squares problem over shared FNO basis functions and computing its own implicit gradient, so the constraint is smaller, parallelizable, and more stable than a single global hard constraint.
Key results
- Diffusion-sorption: PI-HC-MoE relative L2 error 3.60% vs PI-HC 7.55% and PI-SC 85.93%.
- Navier-Stokes (Re 1e4): PI-HC-MoE 8.298% vs PI-HC 11.754% and PI-SC 18.081% relative L2 error on 256x256x64 test set.
- PI-HC-MoE gives 1.613x-3.189x training speedup and 2.030x-3.048x inference speedup over PI-HC on diffusion-sorption; up to 12.838x training speedup on Navier-Stokes at 10^4 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use localized domain-decomposed hard constraints with multiple experts instead of one global hard constraint | FNO neural PDE solver · Diffusion-sorption test set | relative L2 error: 7.55%± 8.10% -> 3.60%±2.93% | Helped |
| Use localized domain-decomposed hard constraints with multiple experts instead of one global hard constraint | FNO neural PDE solver · 2D Navier-Stokes 256×256×64 test set | relative L2 error: 11.754% ± 2.951% -> 8.298%± 2.345% | Helped |
| Use localized domain-decomposed hard constraints with multiple experts instead of one global hard constraint | FNO neural PDE solver · Diffusion-sorption training and inference | runtime: training: 1.613× to 3.189×; inference: 2.030× to 3.048× | Helped |
| Use localized domain-decomposed hard constraints with multiple experts instead of one global hard constraint | FNO neural PDE solver · 2D Navier-Stokes training and inference | runtime: training: 2.117× to 12.838×; inference: 2.538× to 12.864× | Helped |
| Use localized domain-decomposed hard constraints with multiple experts instead of one global hard constraint | FNO neural PDE solver · Diffusion-sorption validation, t > 400 s | relative L2 error: 48.047%± 1.23% -> 14.79%±3.00% | Helped |
| Use learned basis functions rather than standard cubic interpolation (ablation) | PI-HC-MoE · Diffusion-sorption test set | relative L2 error | Helped |
Surprising findings
- Soft-constraint PI-SC fails badly on diffusion-sorption (85.93% error) and its PDE residual initially looks low because the prediction trivially satisfies the PDE near zero.
Limitations
- Sensitive to hyperparameters such as number of basis functions, sampled points, and number of experts.
- Performance limited by expressivity of the underlying FNO basis-function network.
- Only spatial/spatiotemporal uniform domain decompositions explored; choice of decomposition is problem dependent.
Experimental setup
Datasets: PDEBench diffusion-sorption initial conditions Gaussian random field initial conditions for 2D Navier-Stokes
Benchmarks: 1D diffusion-sorption test set 2D Navier-Stokes test set at 256x256x64
Metrics: relative L2 error training step time inference step time speedup
Baselines: PI-SC (soft constraint) PI-HC (single hard constraint)
Builds on: Fourier Neural Operator physics-informed neural networks implicit function theorem differentiable optimization DC3
Closest papers by meaning
- Geometric and Physical Constraints Synergistically Enhance Neural PDE Surrogates (ICML 2025 · similarity 0.88)
- Learning a Neural Solver for Parametric PDEs to Enhance Physics-Informed Methods (ICLR 2025 · similarity 0.87)
- Solving Differential Equations with Constrained Learning (ICLR 2025 · similarity 0.86)
- Riemann Tensor Neural Networks: Learning Conservative Systems with Physics-Constrained Networks (ICML 2025 · similarity 0.86)
- Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective (NeurIPS 2025 · similarity 0.86)
- ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.