Research map / Time series / Foundation models for time series
This Time is Different: An Observability Perspective on Time Series Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Toto, a 151M-parameter decoder-only zero-shot time series foundation model for observability metrics, plus the BOOM benchmark of 350M observations from Datadog telemetry.
Problem
Existing time series foundation models generalize poorly to observability metrics, which are highly nonstationary, high-cardinality, multivariate, and heavy-tailed; no large real-world observability benchmark existed.
Approach
A decoder-only TSFM with per-patch causal instance normalization, proportional factorized (time-wise vs variate-wise) attention with an 11:1 block ratio, a Student-T mixture output head, and a composite NLL + robust (Cauchy) loss, trained on a large mix of Datadog observability data, public data, and synthetic data.
Key results
- On BOOM, Toto achieves MASE 0.617 and CRPS 0.375, 13.1% and 12.4% lower than the next best (Moirai Base), with rank 2.369 vs 4.328.
- On GIFT-Eval, Toto reaches rank 5.495 (best), MASE 0.673, and CRPS 0.437 (second-best CRPS) as of May 2025.
- On LSF zero-shot, Toto attains best performance on 8 of 12 metrics and the lowest average MAE (0.324) and MSE (0.304).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Toto instead of Moirai Base | Zero-shot forecasting · Boom | MASE: 0.710 -> 0.617 | Helped |
| Use Toto instead of Moirai Base | Zero-shot forecasting · Boom | CRPS: 0.428 -> 0.375 | Helped |
| Use Toto instead of TimesFM 2.0 | Zero-shot forecasting · Boomlet | MASE: 0.685 -> 0.617 | Helped |
| Use Toto instead of TimesFM 2.0 | Zero-shot forecasting · Boomlet | CRPS: 0.603 -> 0.519 | Helped |
| Use Toto instead of TimesFM 2.0 | Zero-shot forecasting · GIFT-Eval | MASE: 0.680 -> 0.673 | Helped |
| Use Toto instead of TEMPO | Zero-shot forecasting · GIFT-Eval | CRPS: 0.434 -> 0.437 | Hurt |
| Use Toto instead of Moirai Base | Zero-shot forecasting · LSF | MAE: 0.344 -> 0.324 | Helped |
| Use Toto instead of VisionTS | Zero-shot forecasting · LSF | MSE: 0.309 -> 0.304 | Helped |
| Add causal patch-based scaling instead of naive global scaling (ablation) | Toto · Observability validation data | Forecasting performance | Helped |
| Use a Student-T mixture head instead of a single Student-T distribution (ablation) | Toto · Forecasting evaluation | Forecasting accuracy | Helped |
| Add the robust point-prediction loss to the NLL loss (ablation) | Toto · Forecasting evaluation | Point forecasting accuracy | Helped |
| Add the robust point-prediction loss to the NLL loss (ablation) | Toto · Forecasting evaluation | Probabilistic forecasting accuracy | No clear effect |
Surprising findings
- Toto is top on GIFT-Eval despite several competing models having known partial data leakage with the benchmark.
- Existing TSFMs generalize poorly to observability data, with Toto's margin over general-purpose models being substantial on BOOM.
Limitations
- Assumes fixed-interval time series; missing points handled by heuristic imputation.
- Does not incorporate calendar-based features.
- Performance at extreme prediction lengths not studied.
Experimental setup
Datasets: Datadog observability metrics GIFT-Eval Pretrain Chronos datasets TimesFM-style synthetic data kernel_synth_1m
Benchmarks: BOOM BOOMlet GIFT-Eval Long Sequence Forecasting (LSF)
Metrics: MASE CRPS rank MAE MSE
Baselines: Moirai TimesFM 2.0 Chronos-Bolt Timer Time-MoE VisionTS Auto-ARIMA Auto-ETS Auto-Theta DLinear DeepAR TabPFN-TS
Builds on: TimesFM Chronos Moirai RevIN GIFT-Eval
Closest papers by meaning
- Parsimony or Capability? Decomposition Delivers Both in Long-term Time Series Forecasting (NeurIPS 2024 · similarity 0.86)
- SEMPO: Lightweight Foundation Models for Time Series Forecasting (NeurIPS 2025 · similarity 0.86)
- TiRex-2: Generalizing TiRex to Multivariate Data and Streaming (NeurIPS 2026 · similarity 0.86)
- TimeBase: The Power of Minimalism in Efficient Long-term Time Series Forecasting (ICML 2025 · similarity 0.85)
- Kairos: Toward Adaptive and Parameter-Efficient Time Series Foundation Models (NeurIPS 2026 · similarity 0.85)
- Unlocking the Power of Patch: Patch-Based MLP for Long-Term Time Series Forecasting (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.