Research map / Probabilistic and scientific modeling / Dynamical systems
eXponential FAmily Dynamical Systems (XFADS): Large-scale nonlinear Gaussian state-space modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces XFADS, a low-rank structured variational autoencoder for nonlinear Gaussian state-space models that performs approximate variational smoothing via sample-based Gaussian message passing, scaling linearly in state dimension.
Problem
Learning nonlinear Gaussian state-space models with VAEs requires structured posteriors that capture dense covariances while remaining scalable to large latent dimensions and keeping dynamics expressive for accurate forecasting.
Approach
Treat smoothing as filtering with pseudo-observations that depend on current and future data; approximate the predict step by sample-based moment matching and the update step with low-rank natural-parameter updates, so covariance is represented as low-rank plus process noise and the cost is linear in state dimension.
Key results
- On pendulum and bouncing ball, the proposed method consistently outperforms DVBF, DKF and SVAE in forecasting velocity/position R^2 while all methods reconstruct well in context windows.
- On the monkey reaching dataset, the method makes more informative predictions than baselines in terms of R^2 and bits-per-spike over a 500 ms prediction window.
- Empirical wall-clock time scales linearly in state dimension L, whereas a Kalman filter implementation scales cubically.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use XFADS to learn dynamics and forecast future states | Pendulum and bouncing ball · Pendulum and bouncing ball prediction windows | R² | Helped |
| Use XFADS for neural population dynamics forecasting | Motor cortex reaching-task recordings · Motor cortex prediction window | BPS and R² | Helped |
| Use XFADS for neural population dynamics forecasting | DMFC timing interval reproduction recordings · DMFC context and forecast windows | BPS | Mixed |
| Use low-rank rather than dense precision updates (ablation) | XFADS · 50D and 100D simulated linear dynamical systems | ELBO | Hurt |
| Use low-rank rather than diagonal precision updates (ablation) | XFADS · 50D and 100D simulated linear dynamical systems | ELBO | No clear effect |
Surprising findings
- The variational objective loses the property of lower-bounding the data log-marginal likelihood, yet it consistently outperforms approaches using potentially tight lower bounds.
Limitations
- Variational objective is not a lower bound on the log marginal likelihood.
- Parameter-efficiency of low-rank precision update networks decreases when rank becomes moderate.
- Algorithm needs modification when number of samples S exceeds latent dimension L for efficiency.
Experimental setup
Datasets: pendulum bouncing ball monkey motor cortex reaching data (DMFC/M1) DMFC timing interval reproduction data simulated 50D and 100D linear dynamical systems
Benchmarks: pendulum bouncing ball monkey reaching neural recordings DMFC RSG interval task
Metrics: R^2 bits-per-spike negative ELBO wall-clock time
Baselines: DVBF DKF SVAE LFADS
Builds on: structured variational autoencoder (SVAE) deep Kalman filter deep variational Bayes filter LFADS
Closest papers by meaning
- Identifying latent state transitions in non-linear dynamical systems (ICLR 2025 · similarity 0.89)
- Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamics (ICML 2024 · similarity 0.88)
- Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data (ICLR 2025 · similarity 0.88)
- A scalable generative model for dynamical system reconstruction from neuroimaging data (NeurIPS 2024 · similarity 0.88)
- Multi-modal Gaussian Process Variational Autoencoders for Neural and Behavioral Data (ICLR 2024 · similarity 0.87)
- Probabilistic Decomposed Linear Dynamical Systems for Robust Discovery of Latent Neural Dynamics (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.