Research map / Probabilistic and scientific modeling / Dynamical systems
Latent Representation and Simulation of Markov Processes via Time-Lagged Information Bottleneck
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Time-lagged Information Bottleneck (T-IB), an InfoNCE-based objective that learns low-dimensional Markov process representations preserving autoinformation at a chosen lag time while discarding fast superfluous information, enabling accurate large-time-step latent simulation.
Problem
Simulating large Markov systems over long timescales is computationally expensive because accurate integration requires very short time steps; existing time-lagged dimensionality reduction methods (e.g., linear TICA/VAMPNet) struggle to capture slow dynamics in low dimensions.
Approach
Frame latent simulation as variational inference with an upper bound on error given by the autoinformation gap; maximize time-lagged mutual information with an InfoNCE critic (T-InfoMax) and add an information-bottleneck penalty on superfluous information measured via a conditional Flow++ latent transition model (T-IB).
Key results
- On the 2D Prinz process, T-IB yields latent simulations with lower marginal and transition JS divergence than unregularized T-InfoMax and TICA/VAMPNet, even improving over fitting transitions in the original space.
- On Alanine Dipeptide, Chignolin and Villin, T-IB consistently reduces marginal and transition JS error relative to unregularized T-InfoMax and linear methods.
- Training and unfolding a Villin latent simulation with T-IB takes about 6 hours on a single GPU versus about 2-3 months of molecular dynamics on the same hardware.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the T-IB bottleneck regularization to T-InfoMax | T-InfoMax latent simulation · 2D Prinz Potential | marginal JS divergence | Helped |
| Add the T-IB bottleneck regularization to T-InfoMax | T-InfoMax latent simulation · 2D Prinz Potential | transition JS divergence | Helped |
| Add the T-IB bottleneck regularization to T-InfoMax | T-InfoMax latent simulation · Alanine Dipeptide, Chignolin, and Villin | marginal JS divergence | Helped |
| Add the T-IB bottleneck regularization to T-InfoMax | T-InfoMax latent simulation · Alanine Dipeptide, Chignolin, and Villin | transition JS divergence | Helped |
| Use T-IB representations instead of linear correlation-maximization representations | TICA and VAMPNet latent simulations · Alanine Dipeptide, Chignolin, and Villin | marginal JS divergence | Helped |
| Use T-IB representations instead of linear correlation-maximization representations | TICA and VAMPNet latent simulations · Alanine Dipeptide, Chignolin, and Villin | transition JS divergence | Helped |
| Use T-IB representations instead of original-space transitions | Original-space latent simulation baseline · 2D Prinz Potential | marginal JS divergence | Helped |
| Use T-IB representations instead of original-space transitions | Original-space latent simulation baseline · 2D Prinz Potential | transition JS divergence | Helped |
| Use higher-dimensional VAMPNet representations | VAMPNet latent simulation · Molecular simulations | transition JS divergence | Hurt |
Surprising findings
- Linear correlation-maximization methods (TICA, VAMPNet) struggle to extract dynamic information in low dimensions even with non-linear encoders.
- On the 2D Prinz process, latent simulations from T-IB representations improve upon trajectories unfolded by fitting transitions directly in the original observation space.
Limitations
- Modern mutual information lower bounds have high variance for systems with large autoinformation (e.g., chaotic or unstable systems).
- The latent transition model is a fixed flexible class; more flexible transition models could improve fidelity at slower sampling cost.
- Evaluation focuses on representation dynamics; long-range accuracy when only short simulations are available during training is not yet studied.
Experimental setup
Datasets: 2D Prinz potential synthetic process Alanine Dipeptide MD simulations Chignolin MD simulations Villin MD simulations
Benchmarks: 2D Prinz potential Alanine Dipeptide Chignolin Villin
Metrics: autoinformation mutual information between representation and targets (nats) marginal Jensen-Shannon divergence (MJS) transition Jensen-Shannon divergence (TJS)
Baselines: TICA VAMPNet T-InfoMax (unregularized) Latent Space Simulation (LSS)
Builds on: Information Bottleneck InfoNCE VAMPNet Flow++ TICA
Closest papers by meaning
- Learning low-dimensional state embeddings and metastable clusters from time series data (NeurIPS 2019 · similarity 0.87)
- Identifying latent state transitions in non-linear dynamical systems (ICLR 2025 · similarity 0.86)
- Learning invariant representations of time-homogeneous stochastic dynamical systems (ICLR 2024 · similarity 0.86)
- Predicting partially observable dynamical systems via diffusion models with a multiscale inference scheme (NeurIPS 2025 · similarity 0.86)
- Disentangling Continuous-Time Latent Dynamics: Identifiability of Latent SDEs via Diffusion Shifts (NeurIPS 2026 · similarity 0.86)
- Diffusion Models for Attribution (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.