Research map / Time series / Foundation models for time series

Large Pre-trained time series models for cross-domain Time series analysis tasks

NeurIPS 2024 · Time series › Foundation models for time series · approach: LLM-based time series forecasting · 13 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LPTM, a transformer-based multi-domain pre-trained time-series model with learned adaptive segmentation tokenization trained via masked SSL, evaluated on zero-shot/fine-tuned forecasting and classification.

Problem

Pre-training a single time-series foundation model across heterogeneous domains is hard because fixed-length segmentation of series into tokens is inflexible across different sampling rates, dynamics, and noise levels.

Approach

A learned adaptive segmentation module (GRU plus a segment score function) chooses dataset-specific segment boundaries by scoring segments via the SSL loss, so each domain gets its own tokenization; the segments feed a shared transformer pre-trained with random and last-token masking.

Key results

• Zero-shot forecasting: LPTM achieves the lowest RMSE on all listed benchmarks in Table 1 (e.g., Flu-US 1.14 vs 1.29 for Chronos, M4 0.972 vs 1.125).
• Fine-tuned forecasting: LPTM has the best mean rank (2.55) across nine forecasting benchmarks, improving state of the art on ETT1, ETT2, PEMS-Bays, NY-Bike, and M4.
• Classification: LPTM has the highest mean rank over 35 sensor/behavioral datasets and wins on many, though not all, against baselines such as TARNet and TST.

Measured effects reported in the paper

Surprising findings

• Ablation without segmentation (uniform fixed-length tokens) was usually the worst variant, underperforming several baselines.
• Standard deviation across 10 runs is large in the forecasting tables (e.g., Flu-Japan around 23), suggesting high variance in the reported RMSE values.

Limitations

• Only univariate forecasting is evaluated; probabilistic and multivariate forecasts are not addressed.
• The public pre-training corpus is much smaller than the confidential data used by some prior foundation models.

Experimental setup

Datasets: Project Tycho epidemic data ETT PEMS-Bays METR-LA NYC bike and taxi demand Nasdaq and S&P 500 stock prices M4 UEA archive UCI repository CDC FluView NIID Japan influenza

Benchmarks: Flu-US Flu-Japan ETT1 ETT2 PEMS-Bays NY-Bike NY-Taxi Nasdaq M4 UEA/UCI 35 classification datasets

Metrics: RMSE accuracy mean rank training time

Baselines: LLM-Time TimesFM Lag-LLaMA Chronos MOIRAI Informer Autoformer iTransformer PatchTST TS2Vec TS-TCC SimMTM

Builds on: PatchTST REVIN SimMTM TimeLLM

Closest papers by meaning

• Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer (ICML 2025 · similarity 0.90)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.90)
• Empowering Time Series Analysis with Large-Scale Multimodal Pretraining (NeurIPS 2026 · similarity 0.90)
• Timer: Generative Pre-trained Transformers Are Large Time Series Models (ICML 2024 · similarity 0.89)
• Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization (ICML 2025 · similarity 0.89)
• TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting (ICLR 2024 · similarity 0.89)