Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Simulation-Free Training of Neural ODEs on Paired Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Applies flow matching to train neural ODEs on paired data without simulation, learning data and label embeddings jointly with the dynamics so the linear target flow does not cross trajectories.
Problem
Training neural ODEs for deterministic paired-data tasks (regression, classification) is slow due to many ODE solver function evaluations and unstable adjoint gradients, and naive flow matching on paired data yields crossing trajectories that break data-label coupling.
Approach
Perform flow matching in learned embedding space: data and label encoders are trained jointly with the dynamics function under a flow loss plus a label autoencoding loss (with noise injection), so that the predefined linear interpolant yields non-crossing target trajectories enabling simulation-free NODE training.
Key results
- On UCI regression with Euler 1-step, the proposed method achieves lower RMSE than NODE and CARD on most datasets (e.g., Boston 3.52 vs NODE 5.95 and CARD 130.23).
- Proposed method outperforms CARD on MNIST, SVHN, and CIFAR10 classification while requiring a single function evaluation per training step.
- 1-NN accuracy on learned CIFAR10 data embedding is 65.66% versus 88.47% after the learned dynamics, indicating the dynamics function is not collapsing to identity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learn data and label embeddings jointly with the flow-matching loss (ablation) | End-to-End Latent Flow Matching · CIFAR10 training set | training accuracy | Helped |
| Use zero-padding to match data and label dimensions instead of learning encoders with flow loss (ANODE+FM) (ablation) | End-to-End Latent Flow Matching · CIFAR10 training set | training accuracy | Hurt |
| Pretrain separate data and label autoencoders, then fix the embeddings for flow matching (Autoencoder+FM) (ablation) | End-to-End Latent Flow Matching · CIFAR10 training set | training accuracy | Hurt |
| Explicitly sample t=0 during training (ablation) | End-to-End Latent Flow Matching · CIFAR10 | training accuracy | Helped |
| Add noise to the label autoencoding objective (ablation) | End-to-End Latent Flow Matching · CIFAR10 | test accuracy | Helped |
| Use linear rather than convex or concave predefined dynamics (ablation) | End-to-End Latent Flow Matching · UCI Boston | RMSE | Helped |
| Use the learned dynamics function to transform the embedding instead of classifying directly from the data embedding | End-to-End Latent Flow Matching · CIFAR10 | 1-NN accuracy: 65.66% -> 88.47% | Helped |
Surprising findings
- Applying flow matching directly to paired data without learned encoders fails to fit even the training data due to crossing trajectories, even with dimension augmentation.
- Linear dynamics outperformed more complex concave and convex predefined dynamics.
Limitations
- Relies on a fixed, predefined simple dynamics (linear) rather than a learned per-sample dynamics.
- Some NODE baselines failed to train on certain settings, requiring learning-rate adjustments; comparisons depend on tuned baselines.
- Adjoint method used for NODE baselines on SVHN and CIFAR10 may confound comparison.
Experimental setup
Datasets: MNIST SVHN CIFAR10 UCI regression datasets
Benchmarks: MNIST SVHN CIFAR10 UCI regression datasets
Metrics: classification accuracy RMSE NFE training throughput
Baselines: NODE STEER RNODE CARD ANODE+FM Autoencoder+FM
Builds on: Neural ODE flow matching rectified flow ANODE CARD
Closest papers by meaning
- From Fourier to Neural ODEs: Flow Matching for Modeling Complex Systems (ICML 2024 · similarity 0.90)
- Switched Flow Matching: Eliminating Singularities via Switching ODEs (ICML 2024 · similarity 0.90)
- Semi-supervised Learning of Dynamical Systems with Neural Ordinary Differential Equations: A Teacher-Student Model Approach (AAAI 2024 · similarity 0.87)
- Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions (ICLR 2025 · similarity 0.87)
- Learning to Solve Generative ODEs Beyond the Linear Span (NeurIPS 2026 · similarity 0.87)
- Optimal Flow Matching: Learning Straight Trajectories in Just One Step (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.