Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes branched Fourier neural operators (BFNO) as the ODE function of neural ODEs, using dynamic global convolutions with multiple kernels, and reports gains on image classification, time series classification, and FFJORD-based image generation.
Problem
Neural ODEs typically model the hidden-state time derivative with conventional CNNs or FCNs; naively plugging in Fourier neural operators degrades NODE performance on general ML tasks.
Approach
Treat the NODE right-hand side as a differential operator and model it with a branched Fourier neural operator layer that applies L parallel global Fourier-domain convolution kernels and aggregates them with a fully-connected layer, plus a linear skip path.
Key results
- BFNO-NODE reaches 0.4455 test accuracy on STL-10 versus 0.3711 for the best baseline (ANODE), and 0.2890 on CIFAR-100 versus 0.2608 for GHBNODE.
- Latent-BFNO-NODE achieves 0.874 accuracy on HumanActivity, above ACE-Latent-ODE at 0.859.
- FFJORD-BFNO-RNODE reaches 0.88 bits/dim on MNIST and 3.33 on CIFAR-10 versus 0.97 and 3.38 for FFJORD-RNODE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the conventional NODE ODE function with BFNO-NODE | NODE image classifier · MNIST | test accuracy: 0.9834 $\pm$ 0.0004 -> 0.9752 $\pm$ 0.0021 | Hurt |
| Replace the conventional NODE ODE function with BFNO-NODE | NODE image classifier · CIFAR-10 | test accuracy: 0.6264 $\pm$ 0.0015 -> 0.6289 $\pm$ 0.0054 | Helped |
| Replace the conventional NODE ODE function with BFNO-NODE | NODE image classifier · CIFAR-100 | test accuracy: 0.2608 $\pm$ 0.0076 -> 0.2890 $\pm$ 0.0094 | Helped |
| Replace the conventional NODE ODE function with BFNO-NODE | NODE image classifier · STL-10 | test accuracy: 0.3711 $\pm$ 0.0089 -> 0.4455 $\pm$ 0.0029 | Helped |
| Replace Latent-ODE with Latent-BFNO-NODE | Latent-ODE (ODE enc.) · HumanActivity | accuracy: 0.859 $\pm$ 0.011 -> 0.874 $\pm$ 0.004 | Helped |
| Replace Latent-ODE with Latent-BFNO-NODE | Latent-ODE (ODE enc.) · PhysioNet | AUROC: 0.853 $\pm$ 0.003 -> 0.852 $\pm$ 0.001 | Hurt |
| Replace FFJORD-RNODE's ODE function with BFNO | FFJORD-RNODE · MNIST test images | negative log-likelihood (bits/dim): 0.97 -> 0.88 | Helped |
| Replace FFJORD-RNODE's ODE function with BFNO | FFJORD-RNODE · CIFAR-10 test images | negative log-likelihood (bits/dim): 3.38 -> 3.33 | Helped |
| Vary the number of parallel global convolutions; use L=2 rather than too few or too many (ablation) | BFNO-NODE · STL-10 | accuracy | Mixed |
| Replace BFNO with FNO or AFNO (ablation) | NODE ODE function · Image classification, including STL-10 | accuracy | Hurt |
Surprising findings
- On MNIST image classification, AdamNODE (0.9834) beats BFNO-NODE (0.9752), so the method does not win on every dataset.
- Naively adapting FNO and AFNO into the NODE ODE function degrades performance relative to the proposed BFNO design.
Limitations
- Some reported gains are small or inconsistent, e.g., PhysioNet AUROC is below ACE-Latent-ODE and MNIST accuracy is below AdamNODE.
- Hyperparameters (L, N) were tuned per dataset and the ablations are limited to STL-10 in the main text.
- Experiments are small-scale with fixed parameter-matched NODE architectures; generalization beyond these benchmarks is not shown.
Experimental setup
Datasets: MNIST CIFAR-10 CIFAR-100 STL-10 HumanActivity PhysioNet
Benchmarks: MNIST CIFAR-10 CIFAR-100 STL-10 HumanActivity PhysioNet
Metrics: test accuracy AUROC negative log-likelihood (bits/dim) NFE train loss
Baselines: NODE ANODE SONODE HBNODE GHBNODE NesterovNODE GNesterovNODE AdamNODE ODE-RNN Latent-ODE ACE-Latent-ODE FFJORD-RNODE
Builds on: neural ODE Fourier neural operator adaptive Fourier neural operator FFJORD HBNODE
Closest papers by meaning
- Derivative-enhanced Deep Operator Network (NeurIPS 2024 · similarity 0.89)
- Component Fourier Neural Operator for Singularly Perturbed Differential Equations (AAAI 2024 · similarity 0.89)
- Neural Operators with Localized Integral and Differential Kernels (ICML 2024 · similarity 0.88)
- In-Context Multi-Operator Learning with DeepOSets (NeurIPS 2026 · similarity 0.87)
- Sensitivity-Constrained Fourier Neural Operators for Forward and Inverse Problems in Parametric Differential Equations (ICLR 2025 · similarity 0.87)
- Improved Operator Learning by Orthogonal Attention (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.