Research map / Time series / Causal and generative time series
SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Series
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SynTSBench, a synthetic-data evaluation paradigm that isolates temporal pattern types, noise/anomaly robustness, cross-variable dependencies, and compares forecasting models against theoretical optima.
Problem
Existing time series forecasting benchmarks use observational data with confounded components and no theoretical performance boundaries, making it hard to attribute model strengths and weaknesses.
Approach
Generate programmable synthetic time series with known mathematical optima (trend, periodic, ARMA, random-walk, cross-variable, noise/anomaly-corrupted signals) so model predictions can be compared directly to theoretical boundaries and capabilities can be mapped per pattern type.
Key results
- On SNR -10dB noise, TimesNet achieves the lowest MSE_True (0.0254) among deep models, with PaiFilter and TexFilter closely following, while the theoretical optimum is 0.
- On cross-variable lag experiments, TSMixer and TimesNet show roughly 50% lower MSE on the lagged variable, indicating use of cross-variable structure, whereas channel-independent PaiFilter, TexFilter and PatchTST do not.
- Zero-shot results show Chronos best on trend patterns (MSE 0.0002) while TimeMoE is best on periodic patterns (MSE 0.0594).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TimesNet instead of the other evaluated models under SNR -10dB | time-series forecasting on synthetic noisy signals · SNR -10dB, H=96 | MSE_True: 0.0300 -> 0.0254 | Helped |
| Use DLinear instead of other deep models for ARMA(1,1) forecasting | deep learning forecasting models · ARMA(1,1), H=10 | MSE: 7.6396 -> 6.4687 | Helped |
| Use DLinear for the lag-50 ARMA long-dependency pattern | deep learning forecasting models · ARMA-Long-Dependency, H=10 | MSE: 2.0083 -> 1.0402 | Helped |
| Use TimesNet to model cross-variable lag relationships | multivariate forecasting models · Feature Lag (48 steps), H=96 | MSE: 1.0450 -> 0.7519 | Helped |
| Use Chronos for zero-shot trend forecasting rather than Moirai | time-series foundation models · Trend Patterns | MSE: 0.9332 -> 0.0002 | Helped |
| Use TimeMoE for zero-shot periodic forecasting rather than Chronos | time-series foundation models · Periodic Patterns | MSE: 0.5293 -> 0.0594 | Helped |
| Remove all normalization layers (ablation) | iTransformer · triangle wave and pulse patterns, H=96 | performance | Helped |
| Use normalization layers on patterns where absolute magnitude is predictive | forecasting models · pulse pattern | pulse amplitude prediction | Hurt |
| Use channel-dependent models to exploit cross-variable relationships | multivariate forecasting models · cross-variable lag, additive, conditional, and nonlinear relationships | MSE: approximately 50% | Helped |
| Increase point-anomaly density for TSMixer | TSMixer · point anomalies | error increase: 342% | Hurt |
| Use filter-based models PaiFilter and TexFilter as dataset length increases | models evaluated across dataset lengths · synthetic datasets of varying lengths | MSE | Helped |
Surprising findings
- Under extreme noise (-10dB), MSE_Obs values converge to about 1 for most models, making the observation-based metric non-discriminative, while MSE_True still separates models.
- Normalization layers appear to strip amplitude information; a normalization-free iTransformer variant substantially outperforms the original on pulse and triangle-wave patterns.
Limitations
- The paper's checklist states no error bars or statistical significance tests were reported.
- Synthetic patterns may not capture the complexity of real-world data; the framework's generality beyond the chosen generators is untested.
- Hardware and training compute details are not reported in the main text.
Experimental setup
Datasets: SynTSBench synthetic trend, periodic, ARMA, random-walk, white-noise, cross-variable datasets Synthetic real-world-like datasets (economic, sensor, traffic, SIR, Lotka-Volterra, etc.)
Benchmarks: SynTSBench trend datasets SynTSBench periodic datasets SynTSBench noise/anomaly tests SynTSBench stochastic dependency datasets SynTSBench cross-variable datasets
Metrics: MSE MAE MSE_Obs MSE_True relative degradation
Baselines: ARIMA mean naive Autoformer PatchTST DLinear iTransformer TimesNet TimeMixer TSMixer PaiFilter TexFilter
Builds on: PatchTST DLinear iTransformer TimesNet Chronos
Closest papers by meaning
- Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts (ICLR 2025 · similarity 0.89)
- ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons (NeurIPS 2024 · similarity 0.88)
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.88)
- Timer: Generative Pre-trained Transformers Are Large Time Series Models (ICML 2024 · similarity 0.88)
- Scaling Law for Time Series Forecasting (NeurIPS 2024 · similarity 0.88)
- TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.