Research map / Probabilistic and scientific modeling / Dynamical systems
Learning invariant representations of time-homogeneous stochastic dynamical systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Deep Projection Networks (DPNets), which learn an invariant subspace representation of time-homogeneous discrete and continuous stochastic dynamics by maximizing a projection-based score with a metric distortion penalty, then perform transfer operator regression on the learned features.
Problem
Learning a state representation that faithfully captures the dynamics of time-homogeneous stochastic dynamical systems, so that transfer operators or generators can be accurately estimated and their spectra interpreted without spurious spectral pollution from metric distortion.
Approach
Cast representation learning as maximizing the HS norm of the projected transfer operator between two neural feature maps, relaxed with a metric distortion penalty (tr(C^2 - C - ln C)) and a differentiable, well-conditioned score that avoids covariance inversion; for continuous time-reversible dynamics, maximize the partial trace of the generator.
Key results
- On the logistic map with r=7, DPNets-relaxed reaches spectral error 0.06 (std 0.05) vs 0.21 for VAMPNets over 20 runs, while DPNets has the smallest optimality gap (0.64 vs 0.97 for VAMPNets).
- On Ordered MNIST, DPNets keeps classification accuracy above 95% over evolution steps while other methods degrade.
- On Chignolin, DPNets estimates a transition time of 17.59 ns versus 40 ns reference, outperforming Nystrom PCR (5.27 ns) and Nystrom RRR (0.89 ns).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DPNets instead of VAMPNets | Logistic Map representation learning · Logistic Map | Optimality Gap: 0.97 (0.22) -> 0.64 (0.01) | Helped |
| Use DPNets instead of VAMPNets | Logistic Map representation learning · Logistic Map | Spectral Error: 0.21 (0.09) -> 0.28 (0.03) | Hurt |
| Use DPNets-relaxed instead of DPNets | Logistic Map representation learning · Logistic Map | Spectral Error: 0.28 (0.03) -> 0.06 (0.05) | Helped |
| Use DPNets-relaxed instead of DPNets | Logistic Map representation learning · Logistic Map | Optimality Gap: 0.64 (0.01) -> 1.19 (0.04) | Hurt |
| Use DPNets instead of Cheby-T | Logistic Map representation learning · Logistic Map | Optimality Gap: 1.24 -> 0.64 (0.01) | Helped |
| Use DPNets instead of NoiseKernel | Logistic Map representation learning · Logistic Map | Optimality Gap: 2.17 -> 0.64 (0.01) | Helped |
| Use DPNets-relaxed instead of Nys-PCR | Chignolin representation learning · Chignolin | P: 7.02 -> 12.84 | Helped |
| Use DPNets-relaxed instead of Nys-PCR | Chignolin representation learning · Chignolin | Transition: 5.27 ns -> 17.59 ns; Reference 40 ns | Mixed |
| Use DPNets-relaxed instead of Nys-PCR | Chignolin representation learning · Chignolin | Enthalpy ΔH: -1.76 kcal/mol -> -1.97 kcal/mol; Reference -6.1 kcal/mol | Mixed |
Surprising findings
- DPNets and VAMPNets failed to converge on Chignolin at large scale, so a GNN-based DPNets-relaxed was used instead.
- The DPNets spectral-error curve on the logistic map converged then diverged, possibly due to overfitting.
Limitations
- The continuous-time score relies on covariance matrix inversion and may be unstable.
- Statistical guarantees are only pointwise for fixed weights; uniform concentration over the network parameters is left open.
- DPNets and unrelaxed DPNets failed to converge on the large-scale Chignolin dataset.
Experimental setup
Datasets: Logistic map (simulated) Schwantes potential Langevin SDE (simulated) Ordered MNIST (constructed from MNIST) 2D fluid flow past cylinder dataset Chignolin molecular dynamics (over 500,000 points)
Benchmarks: Logistic map spectral error and optimality gap Ordered MNIST classification accuracy over time Fluid dynamics relative RMSE forecasting Chignolin metastable states / transition times / enthalpy
Metrics: spectral error optimality gap classification accuracy relative RMSE implied timescale enthalpy
Baselines: VAMPNets Dynamic AE Consistent AE DMD KernelDMD ExtendedDMD Nys-PCR Nys-RRR Cheby-T NoiseKernel
Builds on: VAMPNets Deep-CCA Extended dynamic mode decomposition Eckart-Young-Mirsky theorem
Closest papers by meaning
- When are dynamical systems learned from time series data statistically accurate? (NeurIPS 2024 · similarity 0.89)
- Identifying latent state transitions in non-linear dynamical systems (ICLR 2025 · similarity 0.88)
- DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems (ICML 2024 · similarity 0.87)
- Probabilistic Forecasting with Stochastic Interpolants and Föllmer Processes (ICML 2024 · similarity 0.87)
- Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstruction (NeurIPS 2024 · similarity 0.87)
- Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.