Research map / Time series / Foundation models for time series
A decoder-only foundation model for time-series forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pretrains a 200M-parameter decoder-only patched transformer (TimesFM) on ~100B real and synthetic time-points to act as a zero-shot time-series forecaster.
Problem
Zero-shot point forecasting of unseen time-series across domains, horizons and granularities without per-dataset training, which typically requires dataset-specific supervised models.
Approach
Decoder-only transformer over input patches trained with causal attention and random masking of the first patch so all context lengths are covered; output patches are longer than input patches to reduce autoregressive steps, pretrained on a mix of Google Trends, Wiki pageviews, public datasets and synthetic ARMA/trend/seasonal series.
Key results
- On Monash (18 datasets), TimesFM zero-shot achieves scaled MAE GM 0.6846 versus 0.9715 for llmtime (GPT-3) and 0.7005 for N-BEATS, a more than 25% improvement over llmtime.
- On the 8 ETT evaluation tasks (horizons 96 and 192), TimesFM zero-shot averages MAE 0.36, comparable to supervised PatchTST at 0.37.
- A 200M model is trained on O(100B) time-points in about 2 days on a 16-core TPUv5e setup.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase training FLOPS across model sizes (ablation) | TimesFM · Monash | Scaled MAE (Geometric Mean) | Helped |
| increase output patch length from 8 toward 128 (ablation) | TimesFM · ETT, 512-step forecasting horizon | MAE | Helped |
| vary input patch length; 16 and 32 are best, with errors increasing toward either end (ablation) | TimesFM 70M · Monash | Scaled MAE (Geometric Mean) | Mixed |
| add synthetic pretraining data (ablation) | TimesFM 200M · Monash | MAE | Helped |
| add synthetic pretraining data (ablation) | TimesFM 200M · ETTm 15-minute datasets | MAE | Helped |
| add synthetic pretraining data (ablation) | TimesFM 200M · ETTh hourly datasets | MAE | No clear effect |
| replace TimesFM with pretrained PatchTST(ZS) (ablation) | TimesFM zero-shot forecasting · Monash | Scaled MAE | Hurt |
| replace TimesFM with pretrained PatchTST(ZS) (ablation) | TimesFM zero-shot forecasting · ETT | MAE | No clear effect |
| use TimesFM zero-shot instead of N-BEATS | zero-shot forecasting · Monash | Scaled MAE (Geometric Mean): 0.7005 -> 0.6846 | Helped |
| use TimesFM zero-shot instead of llmtime | zero-shot forecasting · Monash | Scaled MAE (Geometric Mean): 0.9715 -> 0.6846 | Helped |
| use TimesFM zero-shot instead of llmtime | zero-shot forecasting · Darts | Scaled MAE (Geometric Mean): 0.4882 -> 0.5767 | Hurt |
| use TimesFM zero-shot instead of PatchTST(ZS) | zero-shot forecasting · ETT (8 tasks; horizons 96 and 192) | MAE: 0.35 -> 0.36 | Hurt |
Surprising findings
- Pretraining without synthetic data hurts performance notably on 15-minute ETTm datasets, while hourly ETTh shows little difference.
- A PatchTST model pretrained with the same data loader to the same FLOPs performs poorly on Monash, suggesting encoder-decoder style training is less suited for zero-shot.
Limitations
- Univariate only; no covariate or date-feature support in the pretrained model.
- Probabilistic forecasting not explored; point MSE loss only.
- Possible data contamination for llmtime on popular Darts datasets; limited hyperparameter tuning.
Experimental setup
Datasets: Google Trends Wiki Pageviews synthetic ARMA/trend/seasonal series M4 Electricity Traffic Weather Favorita Sales LibCity
Benchmarks: Monash Darts Informer ETT (ETTh1, ETTh2, ETTm1, ETTm2)
Metrics: scaled MAE (geometric mean) MAE msMAPE
Baselines: llmtime N-BEATS PatchTST DeepAR ARIMA FEDFormer Autoformer Informer GPT4TS
Builds on: PatchTST N-BEATS reversible instance normalization llmtime
Closest papers by meaning
- ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data (AAAI 2025 · similarity 0.91)
- TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting (NeurIPS 2025 · similarity 0.89)
- AutoTimes: Autoregressive Time Series Forecasters via Large Language Models (NeurIPS 2024 · similarity 0.89)
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.89)
- Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization (ICML 2025 · similarity 0.88)
- TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.