Research map / Time series / Causal and generative time series
Towards Understanding Evolving Patterns in Sequential Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EvoRate, a mutual-information-based measure of evolving patterns in sequential data, and EvoRateW, an optimal-transport variant for snapshot data without correspondences, validated on synthetic, forecasting, feature selection, video, and evolving domain generalization tasks.
Problem
Identifying and quantifying whether sequential data contains evolving patterns that autoregressive models can exploit, which currently relies on human judgment, and extending this to snapshot data without correspondence across timestamps.
Approach
Estimate sequential mutual information between the next observation and the past k steps using a squared-error critic from an autoregressive model (EvoRate), and when no correspondence exists, build the joint distribution via optimal transport between consecutive timestamps (EvoRateW) to estimate first-order MI.
Key results
- On M4-Weekly, EvoRate is highest for k=270 (2.58) but k=90 gives nearly the same value (2.55) with much lower cost, and forecasting error is comparable.
- On evolving domain generalization, the proposed method reaches 70.3% average accuracy versus 58.6% for the best baseline (DRAIN), and 91.2% on Rotated Gaussian vs 61.0%.
- EvoRateW correctly estimates MI on Gaussian sequences without correspondence where joint and separable critics fail to converge.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed EDG method instead of the strongest listed baseline | EDG classification · Rotated Gaussian target domains | accuracy (%): 61.0 -> 91.2 | Helped |
| Use the proposed EDG method instead of the strongest listed baseline | EDG classification · RMNIST target domains | accuracy (%): 46.4 -> 48.5 | Helped |
| Use the proposed EDG method instead of the strongest listed baseline | EDG classification · PowerSupply target domains | accuracy (%): 71.1 -> 71.3 | Helped |
| Use evolving learning instead of invariant learning | EDG classification · Rotated Gaussian target domains | accuracy (%): 47.5 -> 97.7 | Helped |
| Use evolving learning instead of invariant learning | EDG classification · Portraits target domains | accuracy (%): 85.4 -> 89.1 | Helped |
| Use evolving learning instead of invariant learning | EDG classification · PowerSupply target domains | accuracy (%): 70.8 -> 75.7 | Helped |
| Set the forecasting order to k=90 rather than k=45 (ablation) | M4-Weekly TSF · M4-Weekly | average sMAPE: 7.74 -> 6.28 | Helped |
| Increase forecasting order from k=90 to k=270 (ablation) | M4-Weekly TSF · M4-Weekly | average sMAPE: 6.28 -> 6.28 | No clear effect |
| Increase forecasting order from k=90 to k=270 (ablation) | M4-Weekly TSF · M4-Weekly | EVORATE: 2.55 -> 2.58 | Helped |
| Use the first three features rather than all five features (ablation) | Synthetic feature-selection task · Synthetic sequential data | EVORATE | Helped |
| Add the fourth, redundant feature (ablation) | Synthetic feature-selection task · Synthetic sequential data | EVORATE | Hurt |
| Add the fifth, unrelated noise feature (ablation) | Synthetic feature-selection task · Synthetic sequential data | EVORATE | Hurt |
Surprising findings
- ForeCA (entropy-based forecastability) decreases as sequence length grows even though sMAPE improves, so it is not a good indicator of predictability.
- Allowing the encoder g to be trained jointly with the Wasserstein loss causes representation collapse, so g is pretrained separately with an autoencoder.
Limitations
- No experiments on large language models or NLP tasks due to compute constraints.
- Estimation without correspondence depends on the encoder choice and the assumed Gaussian transition; with identity encoder the estimate is higher than with a learned encoder.
- Several theoretical claims are derived under restrictive assumptions (e.g., Gaussian conditionals, linear transitions).
Experimental setup
Datasets: M4 Crypto Player Trajectory KITTI Rotated Gaussian Circle Sine Rotated MNIST Portraits Caltran PowerSupply synthetic Gaussian sequences
Benchmarks: M4-Weekly M4-Monthly M4-Daily Crypto Player Trajectory KITTI Rotated Gaussian Circle Sine Rotated MNIST Portraits Caltran
Metrics: mutual information (nats) sMAPE RMSE classification accuracy
Baselines: ForeCA MIXUP IRM CORAL DIVA LSSAE DRAIN ERM
Builds on: MINE Wasserstein distance / optimal transport ForeCA Sinkhorn
Closest papers by meaning
- How Patterns Dictate Learnability in Sequential Data (NeurIPS 2025 · similarity 0.91)
- Enhancing Evolving Domain Generalization through Dynamic Latent Representations (AAAI 2024 · similarity 0.85)
- The two clocks and the innovation window: When and how generative models learn rules (NeurIPS 2026 · similarity 0.83)
- TimeES: Probabilistic and Deterministic Time Series Forecasting via Evolutionary Spectra (NeurIPS 2026 · similarity 0.83)
- Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach (NeurIPS 2025 · similarity 0.83)
- Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.