Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Physics-informed neural network training
Improves PINN training through sequential, accelerated and competitive learning.
71 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2021–2026) · an approach in Differential equations and PDEs, keyword cluster “pde · physics · partial differential”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
20 papers in 2023–24 and 44 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 11.2% to 14.7%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2021 · Characterizing possible failure modes in physics-informed neural networks (NeurIPS 2021 · 118 citations)
- 2022 · Spline-PINN: Approaching PDEs without Data Using Fast, Physics-Informed Hermite-Spline CNNs (AAAI 2022 · 60 citations)
- 2023 · Competitive Physics Informed Networks (ICLR 2023 · 10 citations)
- 2024 · Challenges in Training PINNs: A Loss Landscape Perspective (ICML 2024 · 40 citations)
- 2025 · Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective (NeurIPS 2025 · 16 citations)
- 2026 · Fast training of accurate physics-informed neural networks without gradient descent (ICLR 2026 · 2 citations)
Earliest papers here
- Characterizing possible failure modes in physics-informed neural networks (NeurIPS 2021 · 118 citations)
- DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolation (AAAI 2021 · 88 citations)
- Spline-PINN: Approaching PDEs without Data Using Fast, Physics-Informed Hermite-Spline CNNs (AAAI 2022 · 60 citations)
What has been tried
37 papers here report 358 measured changes (a component added, removed or swapped, compared with a baseline): helped 73%, no clear effect 4%, hurt 18%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Increase the batch size | Burgers1d, Gray-Scott, Heat2d-CG, and Poisson2d-C · L2RE | Helped | PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs (NeurIPS 2024 · 28 citations) | |
| use ReLU, Sigmoid, or Sin instead of Wavelet activation | Convection · rMAE | 0.023 vs 1.001; 1.112; 1.074 | Hurt | PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks (ICLR 2024 · 27 citations) |
| use SOAP instead of Adam | Wave, Burgers, Allen-Cahn, and KdV · relative L^2 error | Helped | Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective (NeurIPS 2025 · 16 citations) | |
| feed PDE parameters as coordinates without a separate parameter encoder | Reaction equations, coefficient 1~20 · Abs. err. | 0.0042 -> 0.8975 | Hurt | Parameterized Physics-informed Neural Networks for Parameterized PDEs (ICML 2024 · 6 citations) |
| add region optimization | multiple PDEs and base models · rMSE | Helped | RoPINN: Region Optimized Physics-Informed Neural Networks (NeurIPS 2024 · 6 citations) | |
| Use learned basis functions rather than standard cubic interpolation | Diffusion-sorption test set · relative L2 error | Helped | Scaling physics-informed hard constraints with mixture-of-experts (ICLR 2024 · 3 citations) | |
| Add periodization tricks | NLS · relative error (×10⁻³) | 1.31 -> 1.22 | Helped | Number Theoretic Accelerated Learning of Physics-Informed Neural Networks (AAAI 2025 · 2 citations) |
| Use FP64 instead of FP32 for vanilla PINN training | Convection · rRMSE | 0.7640 -> 0.0072 | Helped | FP64 is All You Need: Rethinking Failure Modes in Physics-Informed Neural Networks (NeurIPS 2025 · 2 citations) |
| set equal direction weights rather than alternative weighting strategies | PINNs with two or three loss terms · test MSE relative improvement over Adam | Helped | ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks (ICLR 2025 · 2 citations) | |
| add sub-sequence alignment | Convection, Reaction, and Wave · rMAE | 0.1436 -> 0.0188; 0.0291 -> 0.0094; 0.0481 -> 0.0197 | Helped | Sub-Sequential Physics-Informed Learning with State Space Model (ICML 2025 · 1 citation) |
| Use the learned spectral-mixture covariance (GM or StM) instead of a plain SE or Matérn kernel | High-frequency and multi-scale PDE benchmarks · relative L2 error | GP solvers using the plain SE and Matérn kernel result in several orders of the magnitude bigger errors | Helped | Solving High Frequency and Multi-Scale PDEs with Gaussian Processes (ICLR 2024 · 1 citation) |
| increase standard integral samples from N=1 to N=100 | Poisson problem with singular charges · solution quality | Helped | Learning from Integral Losses in Physics Informed Neural Networks (ICML 2024 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 28 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- Burgers · 2 of 28 papers
- Allen-Cahn · 2 of 28 papers
Baselines
- PINN · 6 of 28 papers
- SPINN · 4 of 28 papers
- Vanilla PINN · 4 of 28 papers
- Adam · 3 of 28 papers
- PCGrad · 3 of 28 papers
- gPINN · 2 of 28 papers
- RoPINN · 2 of 28 papers
- QRes · 2 of 28 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 64 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 6 papers
- Yale University · 3 papers
- Arizona State University · 3 papers
- Georgia Institute of Technology · 2 papers
- ETHZ - ETH Zurich · 2 papers
- Swiss Federal Institute of Technology · 2 papers
- University of Pennsylvania · 2 papers
- Yonsei University · 2 papers
- University at Buffalo · 2 papers
- State University of New York at Buffalo · 2 papers
Use it
Download BibTeX of 31 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Neural PDE solvers (135 papers)
- Physics-informed operator learning (129 papers)
- Neural operators for general geometries (111 papers)
Elsewhere in the topic
- Neural ODE training and analysis (80 papers)
- Probabilistic inference for ODEs and SDEs (59 papers)
- Structured continuous-depth models (29 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).