Research map / Time series / Foundation models for time series
Synthetic Series-Symbol Data Generation for Time Series Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a series-symbol (S²) synthetic data generator producing 40M time series–symbolic expression pairs and SymTime, a time series foundation model pretrained with masked modeling, symbol MLM, and series-symbol contrastive learning with momentum distillation.
Problem
Time series foundation models suffer from scarce and imbalanced real-world pretraining data (e.g., finance and healthcare underrepresented), limiting generalization.
Approach
Randomly generate multivariate symbolic expressions (binary trees with unary operators and affine transforms) and drive them with mixed-distribution and ARMA inputs to produce unlimited paired series and symbol data, then pretrain a time series encoder jointly with a DistilBERT symbol encoder via contrastive and momentum-distillation objectives.
Key results
- SymTime with 96 look-back achieves average long-term forecasting MSE 0.336 and MAE 0.349, best among compared models in the table.
- Pretraining scale from 0B to 50B on S2 monotonically improves long-term forecasting average MSE from 0.358 to 0.336.
- SymTime reports average classification accuracy of 74.9% on 10 UEA datasets and average anomaly detection F1 of 86.31%, both reported as best.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase S² pretraining data from 0B to 50B (ablation) | SymTime · Long-term forecasting average | MSE: 0.358 -> 0.336 | Helped |
| Increase S² pretraining data from 0B to 50B (ablation) | SymTime · M4 short-term forecasting average | OWA: 0.887 -> 0.849 | Helped |
| Add symbol encoder and series-symbol contrastive learning to temporal MTM learning (ablation) | SymTime · ETTh1 and ETTh2 long-term forecasting | MSE | Helped |
| Add symbol encoder and series-symbol contrastive learning to temporal MTM learning (ablation) | SymTime · Real-Data and synthetic S² pretraining for long-term forecasting | MSE | Helped |
| Replace the SymTime backbone (ablation) | SymTime · Downstream tasks | task performance | No clear effect |
| Remove either of the two contrastive losses (ablation) | SymTime time-series encoder · Series-symbol representation learning | representation clustering | Hurt |
Surprising findings
- Replacing the backbone (e.g., the symbol encoder LLM) has no significant impact on downstream performance, with gains attributed to the dual-modal pretraining paradigm.
Limitations
- Gains on ETT datasets are weaker than GPT4TS for imputation, with stronger effects on high-channel datasets.
- Pretraining compute and hardware details are not reported in the provided text.
Experimental setup
Datasets: S2 synthetic series-symbol dataset Monash ETTh1 ETTh2 ETTm1 ETTm2 Weather ECL Traffic Exchange M4 UEA
Benchmarks: ETTm1 ETTm2 ETTh1 ETTh2 Weather ECL Traffic Exchange M4 UEA SMD MSL
Metrics: MSE MAE OWA SMAPE MASE accuracy F1
Baselines: PatchTST iTransformer TimesNet GPT4TS Time-LLM S2IP-LLM Moirai Timer UniTS Moment
Builds on: ALBEF DistilBERT PatchTST TimesNet benchmark Takens' theorem
Closest papers by meaning
- TSGM: A Flexible Framework for Generative Modeling of Synthetic Time Series (NeurIPS 2024 · similarity 0.86)
- OATS: Online Data Augmentation for Time Series Foundation Models (NeurIPS 2026 · similarity 0.86)
- MOMENT: A Family of Open Time-series Foundation Models (ICML 2024 · similarity 0.85)
- Exploring Representations and Interventions in Time Series Foundation Models (ICML 2025 · similarity 0.85)
- VerbalTS: Generating Time Series from Texts (ICML 2025 · similarity 0.85)
- TimeTok: Granularity-Controllable Time-Series Generation via Hierarchical Tokenization (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.