Research map / Probabilistic and scientific modeling / Dynamical systems
Feedback Favors the Generalization of Neural ODEs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes feedback neural networks that add a state-feedback loop (linear gain, then a domain-randomized neural feedback term) to neural ODEs to correct learned latent dynamics and improve generalization without sacrificing nominal accuracy.
Problem
Neural ODEs generalize poorly when the test-time latent dynamics differ from training (learning residuals), and existing fixes such as domain randomization or scaling networks trade off accuracy on the nominal task.
Approach
Add an internal feedback loop to a neural ODE that corrects the learned dynamics using the deviation between measured and predicted states; use a linear observer-style gain with a Lyapunov convergence guarantee, then learn a nonlinear neural feedback term via domain randomization while freezing the nominal neural ODE, yielding a two-degree-of-freedom network.
Key results
- On the quadrotor Lissajous tracking task with multiple uncertainties, FNN-MPC achieves the best tracking performance among compared MPC variants, per the paper's flight tests.
- Feedback neural networks keep nominal-task accuracy of the neural ODE while improving generalization to randomized spiral cases, where domain randomization alone degrades nominal accuracy.
- The paper reports significant improvements over model-based and learning-based methods on irregular-object trajectory prediction (9 test trajectories).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add feedback to correct the learned dynamics | Neural ODE for irregular object trajectory prediction · Irregular object test trajectories | position prediction error | Helped |
| Add a feedback neural network with multi-step prediction | Quadrotor MPC · Lissajous trajectory tracking under uncertainties and disturbances | RMSE | Helped |
| Add the neural ODE aerodynamic model | Quadrotor MPC · Lissajous trajectory tracking | RMSE | Helped |
| Use single-step rather than multi-step training | MLP-MPC · Quadrotor Lissajous trajectory tracking | RMSE | Hurt |
| Add linear feedback (ablation) | Neural ODE on spiral curves · 12 learning-residual levels | prediction error | Helped |
| Add neural feedback (ablation) | Neural ODE on spiral curves · 12 learning-residual levels | generalization performance | Helped |
| Increase the decay rate from a small value (ablation) | Multi-step feedback prediction · Spiral curve | RMSE | Helped |
| Increase the decay rate beyond its beneficial range (ablation) | Multi-step feedback prediction · Spiral curve | RMSE | Hurt |
| Increase the linear feedback gain (ablation) | Linear feedback neural network · Spiral curve under varied uncertainty | multi-step prediction error | Mixed |
| Train the feedback part through domain randomization while freezing the nominal neural ODE | Feedback neural network · Spiral curve nominal and randomized cases | prediction performance | Helped |
| Apply domain randomization to the neural ODE instead of only to the feedback part | Neural ODE · Spiral curve nominal task | prediction performance | Hurt |
Surprising findings
- Domain randomization applied directly to the neural ODE degrades nominal-task accuracy, motivating the separate feedback learning.
Limitations
- Linear feedback gain and decay rate must be tuned manually.
- Convergence guarantee assumes a bounded learning residual; unbounded residuals are not covered.
- Nonlinear neural feedback is only preliminarily tested.
Experimental setup
Datasets: simulated spiral dynamics open-source irregular object free-flight trajectory dataset simulated quadrotor trajectories
Benchmarks: spiral curve prediction irregular object trajectory prediction (9 test trajectories) quadrotor Lissajous tracking flight tests
Metrics: prediction error (2-norm) RMSE
Baselines: Nomi-MPC Neural-MPC MLP-MPC AdapNN-MPC drag model-based method learning-based acceleration model
Builds on: neural ODE domain randomization Luenberger observer differential flatness-based controller
Closest papers by meaning
- How Deep Do We Need: Accelerating Training and Inference of Neural ODEs via Control Perspective (ICML 2024 · similarity 0.88)
- ControlSynth Neural ODEs: Modeling Dynamical Systems with Guaranteed Convergence (NeurIPS 2024 · similarity 0.86)
- Zero-Shot Transfer of Neural ODEs (NeurIPS 2024 · similarity 0.85)
- Improving Neural ODE Training with Temporal Adaptive Batch Normalization (NeurIPS 2024 · similarity 0.85)
- Symbolic Neural Ordinary Differential Equations (AAAI 2025 · similarity 0.84)
- Semi-supervised Learning of Dynamical Systems with Neural Ordinary Differential Equations: A Teacher-Student Model Approach (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.