Research map / Time series / Foundation models for time series
Sundial: A Family of Highly Capable Time Series Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Sundial, a family of native time series foundation models trained with a flow-matching TimeFlow Loss on a trillion-point corpus (TimeBench), achieving state-of-the-art zero-shot point and probabilistic forecasting.
Problem
Time series foundation models usually rely on MSE/quantile losses or discrete tokenization, which cause mode collapse and limit probabilistic (generative) forecasting; the paper asks how to pre-train Transformers natively on continuous-valued time series with flexible distributions.
Approach
A TimeFlow Loss applies conditional flow-matching to a small MLP head conditioned on each patch's Transformer representation, so the autoregressive model learns per-patch predictive distributions over continuous values without a prior density or discrete tokens; multi-patch prediction, RoPE, Pre-LN, FlashAttention and KV cache are added to the Transformer.
Key results
- Sundial-Large (444M) reaches average MSE 0.331 and MAE 0.369 on ETTm1 zero-shot, and achieves a 7.57% average MSE reduction versus Time-MoE with fewer parameters.
- On GIFT-Eval, Sundial attains the best MASE (0.673) and second-best CRPS (0.472) among all compared models.
- Sundial achieves roughly 35x inference speedup over Chronos on the FEV leaderboard, while ranking second among zero-shot pre-trained models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TimeFlow Loss instead of diffusion loss (ablation) | Sundial · TSLib (six-dataset average) | MSE: 0.314 -> 0.290 | Helped |
| use TimeFlow Loss instead of MSE objective (ablation) | Sundial · TSLib (six-dataset average) | MSE: 0.296 -> 0.290 | Helped |
| use TimeFlow Loss instead of diffusion loss (ablation) | Sundial · GIFT-Eval | CRPS: 0.5340 -> 0.5050 | Helped |
| use TimeFlow Loss instead of MSE loss (ablation) | Sundial · GIFT-Eval | CRPS: 0.6420 -> 0.5050 | Helped |
| add RoPE positional encoding (ablation) | Sundial · TSLib zero-shot forecasting | forecasting performance | Helped |
| use Pre-LN instead of Post-LN (ablation) | Sundial · TSLib zero-shot forecasting | forecasting performance | Helped |
| add FlashAttention (ablation) | Sundial · inference | inference time: 43.6% | Helped |
| add KV Cache (ablation) | Sundial · inference | inference time: 43.6% | Helped |
| add FlashAttention and KV Cache (ablation) | Sundial · TSLib forecasting | forecasting performance | No clear effect |
Surprising findings
- Diffusion-based objective performs notably worse than TimeFlow Loss in zero-shot evaluation, even though both generate samples.
- MSE-trained counterpart produces over-smooth single predictions, which the authors attribute to mode collapse.
Limitations
- Long-context capability for high-frequency data is limited, as the authors note that models should strengthen long-context capacity.
- Pre-training corpus TimeBench is not released, so pre-training is hard to reproduce.
Experimental setup
Datasets: TimeBench Chronos pre-training data LOSTA ERA5 FEV leaderboard datasets (fine-tuning) KernelSynth synthetic data
Benchmarks: Time-Series-Library long-term forecasting (ETTm1, ETTm2, ETTh1, ETTh2, ECL, Weather) GIFT-Eval FEV leaderboard
Metrics: MSE MAE MASE CRPS WQL Rank
Baselines: Time-MoE Timer-XL Moirai Chronos TimesFM PatchTST iTransformer TabPFN
Builds on: flow matching RoPE Pre-LN FlashAttention KernelSynth
Closest papers by meaning
- Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts (ICLR 2025 · similarity 0.88)
- Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach (NeurIPS 2025 · similarity 0.88)
- Timer: Generative Pre-trained Transformers Are Large Time Series Models (ICML 2024 · similarity 0.88)
- Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization (ICML 2025 · similarity 0.87)
- SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Series (NeurIPS 2025 · similarity 0.87)
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.