Research map / Probabilistic and scientific modeling / Dynamical systems
ODEFormer: Symbolic Regression of Dynamical Systems with Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ODEFormer, a transformer that infers multidimensional ODE systems in symbolic form from a single noisy, irregularly sampled trajectory, plus the ODEBench benchmark of 63 ODEs.
Problem
Dynamical symbolic regression: inferring interpretable multidimensional ODE systems from a single observed solution trajectory that may be noisy and irregularly sampled, without prior knowledge of the ground-truth form.
Approach
An encoder-decoder transformer is pretrained end-to-end on ~50M synthetic random ODEs with their integrated trajectories, so it predicts symbolic ODE systems directly from (t, x(t)) at inference without per-system fitting or finite-difference derivative estimates.
Key results
- ODEFormer achieves the highest average reconstruction accuracy on both Strogatz and ODEBench, outperforming GP and regression baselines especially under noise and subsampling.
- ODEFormer runs on the order of seconds per trajectory, versus minutes for most baselines except SINDy.
- Generalization accuracy on ODEBench drops by about half relative to reconstruction accuracy for all models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ODEFormer instead of the benchmarked methods | Dynamical symbolic regression · Strogatz and ODEBench | average accuracy across noise and subsampling levels | Helped |
| Use ODEFormer instead of the benchmarked methods | Dynamical symbolic regression · ODEBench generalization | accuracy | Helped |
| Increase system dimensionality (ablation) | ODEFormer · Synthetic data | accuracy | Hurt |
| Increase the total number of unary and binary operators (ablation) | ODEFormer · Synthetic data | accuracy | Hurt |
| Vary the number of trajectory points (ablation) | ODEFormer · Synthetic data | accuracy | No clear effect |
| Evaluate generalization to a different initial condition rather than reconstruction from the observed initial condition (ablation) | ODEFormer · Synthetic data, 1D | accuracy: 85% -> 60% | Hurt |
| Add noise to the trajectory observations (ablation) | ODEFormer · Synthetic data | accuracy | Hurt |
| Subsample trajectory observations (ablation) | ODEFormer · Synthetic data | accuracy | No clear effect |
| Use two-token, one-token, or two-hot number representations instead of the sign-mantissa-exponent representation (ablation) | ODEFormer · Overall performance | performance | Hurt |
| Use additional parameter optimization (ablation) | ODEFormer · Dynamical symbolic regression | accuracy | Mixed |
| Use ODEFormer instead of PySR | Dynamical symbolic regression · Strogatz and ODEBench reconstruction on clean data | accuracy | Mixed |
| Increase observation noise or subsampling | ODEFormer relative to other methods · Strogatz and ODEBench reconstruction | accuracy | Helped |
Surprising findings
- Generalization accuracy drops by about half relative to reconstruction for all models, meaning many reconstructed ODEs do not match ground truth symbolically.
- ODEFormer is surprisingly insensitive to the number of points in the trajectory.
Limitations
- Only handles first-order ODEs; higher-order systems require finite-difference derivative estimates.
- Assumes all variables are observed; struggles with chaotic systems.
- Uses only a single trajectory per ODE; multi-trajectory aggregation attempts did not work convincingly.
Experimental setup
Datasets: Strogatz (PMLB) synthetic random ODEs
Benchmarks: Strogatz ODEBench
Metrics: R^2 > 0.9 accuracy (reconstruction) generalization accuracy expression complexity inference time
Baselines: AFP FE-AFP EHC EPLEX PySR SINDy FFX ProGED
Builds on: transformer encoder-decoder SINDy PySR ProGED Lample & Charton symbolic integration
Closest papers by meaning
- Active Symbolic Discovery of Ordinary Differential Equations via Phase Portrait Sketching (AAAI 2025 · similarity 0.87)
- Discovering Symmetries of ODEs by Symbolic Regression (AAAI 2025 · similarity 0.87)
- Discovering Physics Laws of Dynamical Systems via Invariant Function Learning (ICML 2025 · similarity 0.86)
- Symbolic Neural Ordinary Differential Equations (AAAI 2025 · similarity 0.86)
- Neural ODE Transformers: Analyzing Internal Dynamics and Adaptive Fine-tuning (ICLR 2025 · similarity 0.86)
- Symmetry-Informed Governing Equation Discovery (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.