Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Neural ODE training and analysis
Study and speed up neural ODEs through adaptive control, theory, and simulation-free training.
80 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Differential equations and PDEs, keyword cluster “ordinary · odes · differential equations”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
20 papers in 2023–24 and 23 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 11.2% to 7.7%.
No longer reported by papers here: MNIST (test accuracy, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Principles of Riemannian Geometry in Neural Networks (NeurIPS 2017 · 38 citations)
- 2018 · Neural Ordinary Differential Equations (NeurIPS 2018 · 558 citations)
- 2019 · Neural Jump Stochastic Differential Equations (NeurIPS 2019 · 108 citations)
- 2020 · How to Train Your Neural ODE: the World of Jacobian and Kinetic Regularization (ICML 2020 · 92 citations)
- 2021 · ECG ODE-GAN: Learning Ordinary Differential Equations of ECG Dynamics via Generative Adversarial Learning (AAAI 2021 · 40 citations)
- 2022 · NODEO: A Neural Ordinary Differential Equation Based Optimization Framework for Deformable Image Registration (CVPR 2022 · 36 citations)
- 2023 · Generalization bounds for neural ordinary differential equations and deep residual networks (NeurIPS 2023 · 8 citations)
- 2024 · Signed Graph Neural Ordinary Differential Equation for Modeling Continuous-Time Dynamics (AAAI 2024 · 14 citations)
- 2025 · Neural Variable-Order Fractional Differential Equation Networks (AAAI 2025 · 9 citations)
- 2026 · Neuro-Symbolic AI for Analytical Solutions of Differential Equations (ICML 2026 · 1 citation)
Earliest papers here
- Principles of Riemannian Geometry in Neural Networks (NeurIPS 2017 · 38 citations)
- Neural Ordinary Differential Equations (NeurIPS 2018 · 558 citations)
- Neural Networks Trained to Solve Differential Equations Learn General Representations (NeurIPS 2018 · 46 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
MNIST · test accuracy (higher is better)
6 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 99.2 | Dissecting Neural ODEs (NeurIPS 2020 · 86 citations) |
| 2024 | 99.53 | How Deep Do We Need: Accelerating Training and Inference of Neural ODEs via Control Perspective (ICML 2024) |
What has been tried
65 papers here report 644 measured changes (a component added, removed or swapped, compared with a baseline): helped 70%, no clear effect 8%, hurt 17%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add intermediate and initial-state feature extraction | PeMSD4 · MAE | 19.32 -> 18.81 | Helped | Signed Graph Neural Ordinary Differential Equation for Modeling Continuous-Time Dynamics (AAAI 2024 · 14 citations) |
| Remove SymNet | DR, KS, and NS, N_tr=100 and N_tr=1000 with σ_n=0% · MSE | Hurt | Symbolic Neural Ordinary Differential Equations (AAAI 2025 · 4 citations) | |
| Vary the number of parallel global convolutions; use L=2 rather than too few or too many | STL-10 · accuracy | Mixed | Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations (AAAI 2024 · 3 citations) | |
| Increase the number of latent parameters beyond about 2 | Vlasov, Burgers, and RDE · relative error | No clear effect | CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations (ICML 2024 · 3 citations) | |
| Use graph Transformers instead of GCN message passing | CIFAR-100 · extrapolation accuracy | Mixed | Architecture-Aware Learning Curve Extrapolation via Graph Ordinary Differential Equation (AAAI 2025 · 2 citations) | |
| Use lower-order IMEX-RK2 instead of higher-order IMEX-RK5 | KSE, grid 64 and grid 512 · training efficiency | ∼2.5 times; a factor of 5 | Helped | Semi-Implicit Neural Ordinary Differential Equations (AAAI 2025 · 1 citation) |
| Use data augmentation feedback | Polar motion · prediction performance | Helped | From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems (ICML 2024 · 1 citation) | |
| enforce topology in NCF | FitzHugh-Nagumo · extrapolation loss | 2.5\tento{-1} ± 3.0\tento{-2} -> 1.9\tento{-3} ± 4.6\tento{-4} | Helped | Neural Conjugate Flows: A Physics-Informed Architecture with Flow Structure (AAAI 2025 · 0 citations) |
| reduce maximum single-step horizon from 1.0 to 0.5 | t = 0.25 · KL | 0.8 ± 0.7 -> 2.4 ± 1.9 | Hurt | Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions (NeurIPS 2025 · 0 citations) |
| Use NCF-t2 instead of NCF-t1 | SM OoD · MSE | 677.2±18.7 -> 2.03±0.12 | Helped | Neural Context Flows for Meta-Learning of Dynamical Systems (ICLR 2025 · 0 citations) |
| Learn data and label embeddings jointly with the flow-matching loss | CIFAR10 training set · training accuracy | Helped | Simulation-Free Training of Neural ODEs on Paired Data (NeurIPS 2024 · 0 citations) | |
| Add only the global Poincaré–Hopf constraint instead of the full global and local regularizer | Landau–Khalatnikov parameter plane · Long-term topological consistency | Helped | Context-Informed Neural ODEs Unexpectedly Identify Broken Symmetries: Insights from the Poincaré–Hopf Theorem (ICML 2025) |
What reviewers will expect
Datasets and baselines named in the 16 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- Cora · 3 of 16 papers
- CoauthorCS · 3 of 16 papers
- Photo · 3 of 16 papers
- Citeseer · 2 of 16 papers
- PubMed · 2 of 16 papers
- CoauthorPhy · 2 of 16 papers
- Ogbn-arxiv · 2 of 16 papers
Baselines
- NODE · 2 of 16 papers
- GCN · 2 of 16 papers
- GAT · 2 of 16 papers
- HGCN · 2 of 16 papers
- GRAND-l · 2 of 16 papers
- GRAND-nl · 2 of 16 papers
- F-GRAND-l · 2 of 16 papers
- F-GRAND-nl · 2 of 16 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 43 papers; from OpenAlex and the papers' affiliations).
- Beijing Jiaotong University · 3 papers
- University of Science and Technology of China · 3 papers
- Anhui University · 3 papers
- Nanyang Technological University · 3 papers
- Fudan University · 3 papers
- University of California, Los Angeles · 2 papers
- Lanzhou University · 2 papers
- National University of Defense Technology · 2 papers
- University of Wisconsin - Madison · 2 papers
- Massachusetts Institute of Technology · 2 papers
Use it
Download BibTeX of 39 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Probabilistic inference for ODEs and SDEs (59 papers)
- Structured continuous-depth models (29 papers)
Elsewhere in the topic
- Neural PDE solvers (135 papers)
- Physics-informed operator learning (129 papers)
- Neural operators for general geometries (111 papers)
- Physics-informed neural network training (71 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).