Research map / Label-efficient and robust learning / Semi-supervised learning
Semi-supervised Learning of Dynamical Systems with Neural Ordinary Differential Equations: A Teacher-Student Model Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TS-NODE, a semi-supervised teacher-student framework for neural ODE modeling of dynamical systems that uses teacher-generated pseudo rollouts and student feedback to correct the teacher.
Problem
Neural ODEs for modeling dynamical systems overfit and generalize poorly when only limited labeled trajectories are available, and standard semi-supervised or augmentation techniques are ill-suited to dynamical data.
Approach
A noisy teacher neural ODE generates pseudo rollouts to train a student neural ODE; the student's error on true labeled trajectories serves as feedback, approximated via a REINFORCE-style, Taylor-expanded loss (student improvement times teacher NLL), to refine the teacher while the teacher also fits the labeled data.
Key results
- On the LK system, TS-NODE reduces local error from 20.93 (baseline) to 12.84 and 100%-horizon rollout error from 7346.09 to 244.88.
- On the pendulum, TS-NODE reduces local error from 40.49 to 5.65 and 100%-horizon rollout error from 507.17 to 1.89.
- On the latent Hopper experiment, latent TS-NODE improves 100-step extrapolation error from 6.62 (baseline) to 5.13 while reconstruction is slightly worse (12.31 vs 12.19).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TS-NODE instead of baseline NODE | Neural ODE · Cubic | Local Error: 39.84 -> 32.77 | Helped |
| Use TS-NODE instead of baseline NODE | Neural ODE · Cubic, 100% rollout | Rollouts Error: 1.08 -> 0.96 | Helped |
| Use TS-NODE instead of baseline NODE | Neural ODE · LK | Local Error: 20.93 -> 12.84 | Helped |
| Use TS-NODE instead of baseline NODE | Neural ODE · LK, 100% rollout | Rollouts Error: 7346.09 -> 244.88 | Helped |
| Use TS-NODE instead of baseline NODE | Neural ODE · Pendulum | Local Error: 40.49 -> 5.65 | Helped |
| Use TS-NODE instead of baseline NODE | Neural ODE · Pendulum, 100% rollout | Rollouts Error: 507.17 -> 1.89 | Helped |
| Disable student feedback to the teacher (ablation) | TS-NODE · LK | Local Error: 12.84 -> 23.60 | Hurt |
| Disable student feedback to the teacher (ablation) | TS-NODE · LK, 100% rollout | Rollouts Error: 244.88 -> 2057.37 | Hurt |
| Train NODE with white-noise augmentation instead of TS-NODE | Neural ODE · LK | Local Error: 12.84 -> 23.78 | Hurt |
| Train NODE with 0.95X rescaling instead of TS-NODE | Neural ODE · LK | Local Error: 12.84 -> 106.97 | Hurt |
| Use latent TS-NODE instead of latent neural ODE | Latent neural ODE · Hopper | Reconstruction Error: 12.19 -> 12.31 | Hurt |
| Use latent TS-NODE instead of latent neural ODE | Latent neural ODE · Hopper, 100 steps extrapolation | Extrapolation Error: 6.62 -> 5.13 | Helped |
Surprising findings
- Too large teacher noise sigma (2.718) raises local error above the baseline (46.10 vs 20.93 on LK), showing excessive noise harms learning.
- Naive augmentations such as rescaling by 0.95x degrade performance badly (LK local error 106.97 vs baseline 20.93).
Limitations
- Pseudo rollouts are generated around the training trajectory in a suboptimal manner without optimizing exploration locations.
- Uncertainty-guided exploration of the state space is left to future work, which may matter in high-dimensional problems.
- Experiments are limited to low-dimensional synthetic systems with a single training trajectory.
Experimental setup
Datasets: Converging cubic system (synthetic) Lotka-Volterra system (synthetic) Simple pendulum (synthetic) 14-dimensional Hopper dataset (simulated)
Benchmarks: Converging cubic system Lotka-Volterra system Simple pendulum Hopper (latent neural ODE)
Metrics: local error (MSE) rollouts error at 5%, 10%, 20%, 50%, 100% horizon reconstruction error extrapolation error
Baselines: Baseline neural ODE White Noise augmentation Re-scale augmentation No Feedback ablation Latent neural ODE with GRU encoder
Builds on: Neural ODE REINFORCE Learning to teach via student feedback (semi-supervised image classification)
Closest papers by meaning
- Semi-Implicit Neural Ordinary Differential Equations (AAAI 2025 · similarity 0.89)
- ControlSynth Neural ODEs: Modeling Dynamical Systems with Guaranteed Convergence (NeurIPS 2024 · similarity 0.88)
- Neural Context Flows for Meta-Learning of Dynamical Systems (ICLR 2025 · similarity 0.88)
- Symbolic Neural Ordinary Differential Equations (AAAI 2025 · similarity 0.87)
- Simulation-Free Training of Neural ODEs on Paired Data (NeurIPS 2024 · similarity 0.87)
- From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.