Research map / Time series / Foundation models for time series

Empowering Time Series Analysis with Large-Scale Multimodal Pretraining

NeurIPS 2026 · Time series › Foundation models for time series · approach: Time-series language reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MM-TS, a large-scale multimodal time series pretraining corpus (time series, derived line-plot images, and LLM-generated endogenous/exogenous news text), and HORAI, a frequency-enhanced multimodal autoregressive foundation model that achieves zero-shot SOTA on forecasting and anomaly detection.

Problem

Time series foundation models are pretrained only on unimodal numerical data, and there is no unified multimodal pretraining paradigm or large-scale multimodal corpus for time series; fusing heterogeneous modalities and generalizing across domains remain open challenges.

Approach

Decompose the series into low-frequency and mid-to-high-frequency bands, align low-frequency components with text and high-frequency components with image embeddings via Flow-Attention, adaptively fuse them, and route tokens in a decoder MoE using both temporal and frequency features, trained autoregressively on the MM-TS corpus.

Key results

• HORAI zero-shot achieves best MSE on 9 of 10 forecasting datasets' averages and wins 15 of 18 forecasting cases, reducing MSE 29.6% vs ROSE.
• On anomaly detection, HORAI zero-shot tops 13 of 15 cases, beating DADA by 13.4% AUC-ROC, 19.5% VUS-ROC, and 19.2% VUS-PR.
• HORAI zero-shot exceeds the full-shot GPT4MTS by 11.4% MSE and TimeVLM by 12.0% MSE in forecasting.

Surprising findings

• Random unrelated text degrades performance even more than removing the text modality on EWJ, which the authors interpret as evidence the model uses semantic content.
• Swin Transformer image encoder performs better than ViT-Base on some datasets, though the authors state ViT and Swin achieve similar overall performance.

Limitations

• Pretraining text relies on GPT-4o generation and filtering, which may introduce hallucinations despite quality checks.
• Evaluation of text quality uses LLM judges without human evaluation reported.
• Some baselines are reported under different protocols (zero-shot vs full-shot), complicating comparison.

Experimental setup

Datasets: MM-TS TimeMMD Monash archive UCR UEA PEMS GDELT

Benchmarks: TimeMMD Agriculture Climate Energy Environment Social Good Traffic EWJ KR MDT Weather

Metrics: MSE MAE AUC-ROC VUS-ROC VUS-PR

Baselines: ChatTime VisionTS ROSE Timer MOIRAI GPT4MTS TATS GPT4TS TimeVLM DADA UniTS LLMMixer

Builds on: Timer Flow-Attention instance normalization Mixture of Experts

Closest papers by meaning

• Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis (NeurIPS 2024 · similarity 0.92)
• Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Series (NeurIPS 2025 · similarity 0.90)
• ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data (AAAI 2025 · similarity 0.90)
• Large Pre-trained time series models for cross-domain Time series analysis tasks (NeurIPS 2024 · similarity 0.90)
• MOMENT: A Family of Open Time-series Foundation Models (ICML 2024 · similarity 0.89)
• Rethinking Post-Training Recipes for Multimodal Time-Series Forecasting (NeurIPS 2026 · similarity 0.89)