Research map / Probabilistic and scientific modeling / Differential equations and PDEs
SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SDE Matching, a simulation-free way to train Latent SDEs by parameterizing the posterior marginals directly through a noise-transform function, avoiding adjoint-based simulation.
Problem
Training Latent SDEs relies on adjoint sensitivity methods that require numerical SDE simulation and backpropagation through solvers, limiting scalability to long and high-dimensional sequences.
Approach
Parameterize the posterior marginals implicitly via an invertible reparameterization F(eps, t, X) so samples z_t are drawn directly without integration; derive the matching conditional ODE and SDE with the same marginals and use a Monte Carlo estimate of the variational bound with one sample per batch.
Key results
- On motion capture, SDE Matching reaches test MSE 4.50 ± 0.32, comparable to adjoint-based Latent SDE (4.03 ± 0.2) and better than ARCTA (7.62 ± 0.93).
- On the Lorenz attractor, SDE Matching needs one drift evaluation per iteration versus 100 simulation steps for adjoint, with per-iteration runtime about 5x faster and over 50x combined speedup.
- On the pendulum video task, SDE Matching learns dynamics in about 20 minutes on a single GPU for 20k iterations, while adjoint takes about 30 hours and fails to capture the dynamics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SDE Matching instead of adjoint sensitivity training | Latent SDE · Motion capture test | Test MSE: 4.03 \pm 0.2 -> 4.50 \pm 0.32 | Hurt |
| Use SDE Matching instead of ARCTA | Latent SDE · Motion capture test | Test MSE: 7.62 \pm 0.93 -> 4.50 \pm 0.32 | Helped |
| Replace adjoint sensitivity training with SDE Matching | Latent SDE · Stochastic Lorenz attractor | training iteration runtime: approximately five times faster | Helped |
| Use SDE Matching rather than adjoint sensitivity training | Latent SDE · Stochastic Lorenz attractor | combined runtime speedup: over $50\times$ | Helped |
| Use SDE Matching rather than adjoint sensitivity training | Latent SDE · Stochastic Lorenz attractor, T=1, 2, 5, 10 | log10 gradient norm: $6.26 \pm 0.14$, $6.95 \pm 0.20$, $7.91 \pm 0.23$ and $8.82 \pm 0.28$ -> $4.92 \pm 0.24$ | Helped |
| Replace adjoint sensitivity training with SDE Matching | Latent SDE · Moving pendulum video, 32 × 32 images | training time: around 30 hours -> approximately 20 minutes | Helped |
| Parameterize posterior marginals directly with a reparameterization function | SDE Matching · Stochastic Lorenz attractor | convergence speed: over $50\times$ | Helped |
| Use state-dependent instead of state-independent volatility (ablation) | SDE Matching · Stochastic Lotka–Volterra system | trajectory structure | Helped |
Surprising findings
- Adjoint-method gradient norms grow exponentially with time horizon, while SDE Matching keeps gradient norms stable across horizons.
Limitations
- Posterior parameterization F must be invertible and provide a tractable score, restricting flexibility.
- Evaluating the drift correction term is costly for high-dimensional general-form diffusion terms.
- Method is only validated on modest-scale benchmarks; not compared to large-scale generative sequence models.
Experimental setup
Datasets: 3D stochastic Lorenz attractor (synthetic) CMU motion capture subject 35 pendulum video (32x32) stochastic Lotka-Volterra (synthetic)
Benchmarks: motion capture test MSE 3D stochastic Lorenz attractor pendulum video
Metrics: test MSE training time per iteration convergence gradient norm
Baselines: Latent SDE (adjoint sensitivity) ARCTA Latent ODE NeuralODE ODE2VAE
Builds on: Latent SDE adjoint sensitivity method score-based generative modeling flow matching ARCTA
Closest papers by meaning
- Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions (NeurIPS 2025 · similarity 0.90)
- Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions (ICLR 2025 · similarity 0.90)
- Trajectory Flow Matching with Applications to Clinical Time Series Modelling (NeurIPS 2024 · similarity 0.90)
- SING: SDE Inference via Natural Gradients (NeurIPS 2025 · similarity 0.88)
- Closing the Approximation Gap in Simulation-free Latent SDEs (NeurIPS 2026 · similarity 0.87)
- Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.