Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Mechanistic Neural Networks for Scientific Machine Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Mechanistic Neural Networks, which embed learnable ODE representations (Mechanistic Blocks) solved by a GPU-parallel relaxed linear-programming solver (NeuRLP), and applies them to equation discovery, PDE solving, N-body, parameter discovery and time series.
Problem
Scientific machine learning needs models that explicitly learn governing differential equations from data while remaining efficient to train, whereas sequential ODE solvers and symbolic regression (SINDy) are slow for batched training or restricted to linear combinations of basis functions.
Approach
A mechanistic encoder outputs coefficients of a family of ODEs per input, which are solved in a single differentiable layer by NeuRLP: a linear-program formulation of ODE solving, relaxed to an equality-constrained quadratic program solved via a batched KKT system on GPU, with learnable step sizes and initial conditions.</120>
Key results
- On 2-d Darcy Flow, Mechanistic NN reaches RMSE 0.0065 versus 0.0070 for FNO, the best baseline in the table.
- On N-body ephemerides, Mechanistic NNs improve over Neural ODEs by at least a factor of 10 in evaluation loss, as stated in the text.
- NeuRLP solves 1000-step noisy sinusoid fitting in 3.68 s versus 666.69 s for torchdiffeq RK4 per 100 iterations, with lower MSE (292 vs 589).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Mechanistic NN instead of FNO | 2d Darcy Flow modeling · 2d Darcy Flow | RMSE: 0.0070 -> 0.0065 | Helped |
| Use NeuRLP (QP) instead of RK4 | Noisy sinusoidal wave fitting · Noisy sinusoidal waves, 1000 steps | MSE loss: 589 -> 292 | Helped |
| Use NeuRLP (QP) instead of RK4 | Noisy sinusoidal wave fitting · Noisy sinusoidal waves, 1000 steps | time (seconds): 666.69 -> 3.68 | Helped |
| Use NeuRLP instead of the torchdiffeq RK4 adjoint solver | Long and noisy sinusoidal fitting · 1000 steps | time (speed): about 200x faster | Helped |
| Use Mechanistic NN instead of Neural ODE | N-body prediction · Solar system ephemerides | evaluation loss: at least a factor of 10 | Helped |
| Use Mechanistic NN instead of second-order Neural ODE | Airplane vibration forecasting · Aircraft shaker accelerations | training error: two times lower training error | Helped |
Surprising findings
- On the airplane vibration task, Mechanistic NNs are only on-par with second-order Neural ODEs in final accuracy, despite faster convergence.
- The authors note they cannot guarantee identifiability of the discovered ODEs, though the recovered equations often lie close to the true ones.
Limitations
- Identifiability of computed ODEs is not measured or guaranteed.
- Model design space was not explored; better architectures may exist.
- Memory for storing n x (d+2) ODE coefficients grows with grid size and ODE order.
Experimental setup
Datasets: Lorenz system trajectories 1-d KdV equation 2-d Darcy Flow JPL Horizons planetary ephemerides aircraft vibration benchmark nested spheres synthetic two-body problem
Benchmarks: Lorenz system discovery KdV 1-d PDE 2-d Darcy Flow JPL Horizons N-body ephemerides aircraft vibration nested spheres scipy odeint linear ODE validation
Metrics: MSE relative MSE RMSE evaluation loss cosine similarity of forces computation time
Baselines: SINDy Neural ODE Second-order Neural ODE Augmented Neural ODE FNO Lie-group augmented ResNet RK4 (torchdiffeq)
Builds on: Neural ODE SINDy Fourier Neural Operator differentiable optimization (OptNet)
Closest papers by meaning
- Scalable Mechanistic Neural Networks (ICLR 2025 · similarity 0.88)
- Mechanistic PDE Networks for Discovery of Governing Equations (ICML 2025 · similarity 0.86)
- PRDP: Progressively Refined Differentiable Physics (ICLR 2025 · similarity 0.84)
- Breakeven complexity: A new perspective on neural partial differential equation solvers (NeurIPS 2026 · similarity 0.84)
- Neural Oscillators for Generalization of Physics-Informed Machine Learning (AAAI 2024 · similarity 0.83)
- From Neurons to Neutrons: A Case Study in Interpretability (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.