Research map / Probabilistic and scientific modeling / Dynamical systems
SING: SDE Inference via Natural Gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SING, a natural-gradient variational inference method for latent SDE models that uses associative scans to parallelize updates over time, with a convergence bound to the continuous-time ELBO.
Problem
Posterior inference over latent state trajectories in latent SDE models is intractable for non-conjugate models, and existing variational methods converge slowly and are numerically unstable.
Approach
Applies natural gradient VI to a linear time-varying SDE (Gaussian Markov) variational family on a discretized ELBO, using Stein's lemma to reduce expectations of nonlinear prior transitions to R^D and associative scans to compute the log normalizer in O(D^3 log T) parallel time; also extends to GP-SDE drifts (SING-GP).
Key results
- SING recovers the true posterior of a linear Gaussian latent SDE in a single iteration, while VDP needs several iterations and Adam is outperformed.
- Parallel SING is roughly 100x faster than sequential SING at T=4096 on a single trial on an NVIDIA A100.
- SING-GP is robust to discretization step size on mouse hypothalamus data, while VDP-GP degrades at large Δt.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SING instead of VDP, Adam, or CMGS for latent inference | Place cell model · Place cell model | latent inference accuracy and convergence speed | Helped |
| Use SING instead of VDP for variational inference | Duffing drift estimation · Duffing synthetic data | latent RMSE and dynamics RMSE | Helped |
| Use parallel rather than sequential natural-to-mean conversion (ablation) | SING · LDS with Gaussian observations, single trials | runtime: roughly $100 \times$ faster for $T = 4096$ | Helped |
| Use Monte Carlo rather than quadrature to approximate expectations (ablation) | SING · Embedded Lorenz attractor, $D \le 5$ | latent RMSE | No clear effect |
| Decrease the time discretization step (ablation) | SING and VDP · Duffing drift estimation | dynamics recovery accuracy | Helped |
| Use a larger discretization step (ablation) | SING-GP rather than VDP-GP · Neural dynamics during aggression | expected log-likelihood and convergence speed | Mixed |
| Use SING-GP rather than VDP-GP | GP-SDE model · Neural dynamics during aggression | forward-simulation predictive $R^2$ | Helped |
| Use SING rather than SDE Matching | Latent inference · Lorenz attractor | ground-truth latent recovery | Helped |
| Use SING rather than SDE Matching | Drift inference · Lorenz attractor | drift recovery | Hurt |
| Use a GP drift prior rather than a parametric drift class (ablation) | SING-GP · Duffing synthetic data | latent RMSE and dynamics RMSE | No clear effect |
| Use a GP drift prior (ablation) | SING-GP · Duffing synthetic data | drift uncertainty estimation | Helped |
Surprising findings
- A single Monte Carlo sample per expectation matches quadrature accuracy for latent dimensions D<=5 and remains accurate up to 50-dimensional latents.
Limitations
- Discretization grid is fixed a priori rather than adaptively learned
- Inducing point locations in SING-GP are fixed rather than learnable
Experimental setup
Datasets: synthetic linear spiral SDE place cell Van der Pol simulation embedded Lorenz attractor Duffing oscillator mouse ventromedial hypothalamus calcium imaging
Benchmarks: synthetic latent SDE simulations mouse hypothalamus neural data
Metrics: latents RMSE drift RMSE expected log-likelihood runtime forward simulation R^2
Baselines: Kalman smoothing VDP Adam on natural parameters CMGS SDE Matching VDP-GP quadrature
Builds on: natural gradient variational inference VDP (variational inference for diffusion processes) GP-SDE Verma et al. 2024 NGVI for latent SDEs
Closest papers by meaning
- Closing the Approximation Gap in Simulation-free Latent SDEs (NeurIPS 2026 · similarity 0.92)
- Variational Inference for SDEs Driven by Fractional Noise (ICLR 2024 · similarity 0.91)
- In-Context Learning of Stochastic Differential Equations with Foundation Inference Models (NeurIPS 2025 · similarity 0.88)
- SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations (ICML 2025 · similarity 0.88)
- Globally Convergent Variational Inference (NeurIPS 2024 · similarity 0.88)
- Brain-like Variational Inference (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.