Research map / Time series / Causal and generative time series

Curriculum Multiple Shooting for Robust Training of Neural and Universal Differential Equations

NeurIPS 2026 · Time series › Causal and generative time series · approach: Neural differential equations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes curriculum multiple shooting (CMS), which starts from a multiple-shooting objective and progressively merges windows until recovering single shooting, to train NODEs, UDEs and mechanistic ODEs more robustly.

Problem

Training neural ODEs and universal differential equations on noisy, sparse, partially observed time-series data is difficult; existing strategies (single shooting, multiple shooting, curriculum learning, collocation) trade off applicability against practicality and tuning burden.

Approach

Combine curriculum learning with multiple shooting by starting from many short overlapping windows and progressively merging adjacent windows across curriculum stages until a single full-trajectory window (single shooting) is reached, with a continuity penalty on shared measurement points.

Key results

• On the Lotka-Volterra UDE, CMS achieved median extrapolation MSE 0.0148 versus 0.0251 for multiple shooting (p=1.3e-3).
• On the SIR UDE, CMS extrapolation MSE was 0.0606 versus 24.9 for multiple shooting, and matched the best method CL (0.0605).
• On eight PEtab mechanistic ODE models, CMS outperformed the single-shooting control on all models with an average fold-change of 8.9 in runs converging to the global optimum, and was best on 5/8 models.

Surprising findings

• Adam+BFGS achieved the lowest interpolation MSE on the Lotka-Volterra UDE but extrapolated much worse (1.52 vs 0.0148 for CMS), indicating overfitting.
• CMS allowed stronger output regularization without increasing interpolation MSE, recovering mechanistic parameters close to ground truth where MS drove them to zero.

Limitations

• Only a limited number of NODE and UDE benchmarks (four) were considered.
• Models are low-dimensional (<20 states); scalability to high-dimensional data is untested.
• CMS introduces several tuning parameters (learning rate, number of windows, continuity penalty), though defaults are proposed.

Experimental setup

Datasets: simulated Lotka-Volterra data UK COVID-19 case data (JHU CSSE) double pendulum data PEtab benchmark collection (8 mechanistic ODE models)

Benchmarks: Lotka-Volterra UDE Lotka-Volterra NODE SIR UDE double pendulum UDE PEtab mechanistic ODE benchmarks (8 models) Julia SciML UDE/NODE benchmarks

Metrics: median interpolation MSE median extrapolation MSE number of multistarts converging to global optimum Wilcoxon rank-sum test with Holm correction

Baselines: Adam curriculum learning (CL) multiple shooting (MS) Adam+BFGS Fides (single shooting control)

Builds on: multiple shooting curriculum learning universal differential equations neural ODE PEtab

Closest papers by meaning

• Differentiable Multiple Shooting Layers (NeurIPS 2021 · similarity 0.88)
• Simulation-Free Training of Neural ODEs on Paired Data (NeurIPS 2024 · similarity 0.86)
• Semi-supervised Learning of Dynamical Systems with Neural Ordinary Differential Equations: A Teacher-Student Model Approach (AAAI 2024 · similarity 0.86)
• Global Convergence in Neural ODEs: Impact of Activation Functions (ICLR 2025 · similarity 0.86)
• How Deep Do We Need: Accelerating Training and Inference of Neural ODEs via Control Perspective (ICML 2024 · similarity 0.86)
• Neural Context Flows for Meta-Learning of Dynamical Systems (ICLR 2025 · similarity 0.86)