Research map / Probabilistic and scientific modeling / Differential equations and PDEs
SHoP: A Deep Learning Framework for Solving High-Order Partial Differential Equations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SHoP, a framework that derives an explicit matrix-form rule for high-order derivatives of MLPs and uses it to solve high-order PDEs and expand trained networks into Taylor polynomials.
Problem
Neural-network PDE solvers built on autograd become intractable in memory and time for high-order derivatives, and the learned network is a black box with no explicit form.
Approach
Derives a recurrence-based transformation matrix (chain rule for composite functions extended to MLPs) that yields all derivatives up to order n in one pass, and expands the trained MLP into an explicit Taylor series; also proves Taylor-series equivalence and gives a convergence condition tied to weight magnitude.
Key results
- On a 2D 10-order derivative benchmark with p=2, SHoP takes 0.3828 s versus 1435.0 s for Autograd.
- On the 3D 4th-order heat equation, SHoP reaches R2 of 0.9998 versus 0.9962 for PINN.
- Autograd runs out of memory for p=3 and n>=8, while SHoP finishes p=3, n=8 in 0.2618 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SHoP's high-order derivative rule instead of Autograd | MLP derivative calculation · p=2, n=10 derivatives | running time: 1435.0 -> 0.3828 | Helped |
| Use SHoP's high-order derivative rule instead of Autograd | MLP derivative calculation · p=3, n=8 derivatives | running time: OOM -> 0.2618s | Helped |
| Use SHoP instead of PINN | PDE solution · 2D 8th-order Helmholtz equation | running time: 1.18e5 seconds -> 834 seconds | Helped |
| Use SHoP instead of PINN | PDE solution · 3D 4th-order Heat equation | R2: 0.9962 -> 0.9998 | Helped |
Limitations
- Derivation assumes all components are differentiable; ReLU/LeakyReLU only yield first-order derivatives.
- Derived only for basic MLPs; CNN, RNN, and transformer extensions are not carried out.
- Convergence of the Taylor series depends on weight magnitude, so constraints must be imposed to use it.
Experimental setup
Benchmarks: 1D 4th-order harmonic oscillator 2D 4th-order biharmonic equation 2D 8th-order Helmholtz equation 3D 4th-order heat equation
Metrics: coefficient of determination R2 running time
Baselines: PINN Autograd
Builds on: PINN automatic differentiation Adamax
Closest papers by meaning
- Breakeven complexity: A new perspective on neural partial differential equation solvers (NeurIPS 2026 · similarity 0.86)
- Text2PDE: Latent Diffusion Models for Accessible Physics Simulation (ICLR 2025 · similarity 0.86)
- Derivative-enhanced Deep Operator Network (NeurIPS 2024 · similarity 0.86)
- Mechanistic PDE Networks for Discovery of Governing Equations (ICML 2025 · similarity 0.85)
- Unisolver: PDE-Conditional Transformers Towards Universal Neural PDE Solvers (ICML 2025 · similarity 0.85)
- An Interpretable Approach to the Solutions of High-Dimensional Partial Differential Equations (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.