Research map / Time series / Foundation models for time series
True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DynaMix, a ~10k-parameter mixture-of-experts of AL-RNNs pretrained on 34 synthetic 3d dynamical systems, which zero-shot reconstructs unseen dynamical systems and empirical time series from a context signal, outperforming TS foundation models on long-term statistics.
Problem
Zero-shot dynamical systems reconstruction (DSR): generating surrogate models of unseen dynamical systems from a short context signal that reproduce long-term attractor geometry and temporal statistics, which existing DSR methods (purpose-trained) and time series foundation models (short-term focuse
Approach
A gated mixture of multivariate AL-RNN experts, where a state-attention and CNN-based gating network uses the context signal to select expert mixtures, pretrained with sparse teacher forcing on many synthetic DS so it can infer new systems in-context without retraining.
Key results
- DynaMix achieves the best median D_stsp and D_H against TS foundation models across 54 test DS with context length 2000, with Wilcoxon-tested significance.
- DynaMix uses about 0.1% of the parameters (~10k) of the Chronos models and yields inference times about an order of magnitude faster than the closest competitor.
- On Lorenz-63 Chronos gives max Lyapunov exponent ~0.02 (periodic-like) while DynaMix gives ~0.96 versus ground truth ~0.87.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DynaMix instead of the strongest tested time-series foundation models | Zero-shot DSR across 54 test DS · 54 test DS | D_stsp and D_H | Helped |
| Use DynaMix instead of the strongest tested time-series foundation models | Zero-shot 10-step forecasting across 54 test DS · 54 test DS | MASE | Mixed |
| Add sparse teacher forcing (STF) instead of standard BPTT training (ablation) | DynaMix · Zero-shot DSR | DSR quality | Helped |
| Train on a diverse corpus instead of Lorenz-type systems only (ablation) | DynaMix · Zero-shot out-of-domain DSR | DSR quality | Helped |
| Use a sufficient number of experts instead of too few experts (ablation) | DynaMix · Zero-shot DSR | DSR quality | Helped |
| Add the attention mechanism to the gating network (ablation) | DynaMix · Zero-shot DSR | DSR quality | Helped |
| Add CNN context preprocessing instead of removing the CNN (ablation) | DynaMix · Zero-shot DSR | DSR quality | Helped |
| Replace CNN preprocessing with a linear layer (ablation) | DynaMix · Zero-shot DSR | DSR quality | Hurt |
| Replace the gating MLP with a simple linear operation (ablation) | DynaMix · Zero-shot DSR | DSR quality | No clear effect |
| Replace AL-RNN experts with clipped shallow PLRNN experts (ablation) | DynaMix · Zero-shot DSR | DSR quality | No clear effect |
| Add noise to the AL-RNN experts to make them probabilistic (ablation) | DynaMix · Zero-shot DSR | Long-term performance | Helped |
| Add noise to the AL-RNN experts to make them probabilistic (ablation) | DynaMix · Zero-shot forecasting | Short-term prediction accuracy | Hurt |
Surprising findings
- Chronos produces context parroting that yields cyclic repetition, missing chaotic dynamics on Lorenz-63 despite low short-term error.
- DynaMix often outperforms TS foundation models on MAE for empirical time series (traffic, weather, fMRI, EEG) despite never being trained on empirical data.
Limitations
- Mainly evaluated on rather stationary time series; non-stationarity, tipping points and multiscale data are difficult.
- Incorrect embeddings or sharp-peaked slow series can cause forecast failures.
- Trained on a narrow corpus of 34 mostly 3d chaotic/cyclic synthetic systems; continuous-time / irregular sampling not handled.
Experimental setup
Datasets: 34 simulated chaotic and cyclic 3d dynamical systems Lorenz-63 Lorenz-96 Selkov Van-der-Pol Rayleigh oscillator traffic Huawei cloud requests ETTh1 weather (soil temperature, air pressure) fMRI EEG
Benchmarks: 54 test DS (3d) Lorenz-63 Lorenz-96 Selkov traffic Huawei cloud requests ETTh1 weather fMRI EEG
Metrics: D_stsp (KL divergence in state space) D_H (Hellinger distance of power spectra) MASE MAE maximum Lyapunov exponent
Baselines: Chronos Chronos-2 Panda Mamba4Cast TimesFM Tiny Time Mixers AL-RNN (custom-trained) Neural ODE reservoir computer
Builds on: AL-RNN sparse teacher forcing Chronos PLRNN
Closest papers by meaning
- A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems (NeurIPS 2026 · similarity 0.92)
- A scalable generative model for dynamical system reconstruction from neuroimaging data (NeurIPS 2024 · similarity 0.89)
- Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data (ICLR 2025 · similarity 0.89)
- Out-of-Domain Generalization in Dynamical Systems Reconstruction (ICML 2024 · similarity 0.88)
- Zero-shot forecasting of chaotic systems (ICLR 2025 · similarity 0.87)
- Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamics (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.