Research map / Time series / Foundation models for time series
Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Time-MMD, a multi-domain numerical-text time series dataset across 9 domains with contamination-aware text preprocessing, and MM-TSFlib, a library that extends unimodal TSF models with frozen LLM text embeddings for multimodal forecasting.
Problem
Existing time series analysis models are unimodal, and existing multimodal time series datasets are narrow in domain, coarsely aligned with text, and contaminated by predictions and LLM pretraining exposure.
Approach
Build a multi-domain dataset with fine-grained numeric-text alignment via binary timestamps, use LLM (Llama3-70B) to filter text and separate facts from predictions, and provide a modular framework that fuses frozen LLM text embeddings with any TSF backbone through learnable linear weighting.
Key results
- Multimodal versions outperformed unimodal counterparts in 95% of over 1,000 experiments, with over 15% average MSE reduction.
- MSE reductions reach up to 40% in domains with rich textual data.
- Performance of multimodal TSF is similar across LLM backbones, with no clear scaling law for GPT-2 variants.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add textual data to a unimodal TSF model | TSF backbones evaluated with MM-TSFlib · Time-MMD, all 9 domains | MSE: over 15% | Helped |
| Add textual data to a unimodal TSF model | TSF backbones evaluated with MM-TSFlib · Time-MMD domains with rich textual data | MSE: up to 40% | Helped |
| Vary the LLM backbone (ablation) | MM-TSFlib multimodal TSF · Health | MSE | No clear effect |
| Scale up the GPT-2 backbone (ablation) | MM-TSFlib multimodal TSF · Health | MSE | No clear effect |
| Replace BERT with Doc2Vec for text modeling (ablation) | MM-TSFlib multimodal TSF · Three datasets across four prediction horizons | MSE | Hurt |
| Increase the forecast horizon (ablation) | MM-TSFlib multimodal TSF · Time-MMD domains | MSE reduction from multimodal extension | No clear effect |
Surprising findings
- The choice of LLM backbone (including Llama-3-8B vs. BERT) shows no significant correlation with multimodal TSF performance.
- Scaling GPT-2 size does not clearly improve multimodal forecasting performance.
Limitations
- All text data is in English, limiting linguistic diversity.
- Framework only trains a projection layer on frozen LLMs, not an efficient or optimal fine-tuning approach.
- Only forecasting is studied; multivariate forecasting and classification are not covered.
Experimental setup
Datasets: Time-MMD
Benchmarks: Time-MMD
Metrics: mean squared error
Baselines: Transformer Reformer Informer Autoformer Crossformer Non-stationary Transformer FEDformer iTransformer DLinear FiLM Time-LLM PatchTST
Builds on: Time-Series Library Time-LLM Doc2Vec
Closest papers by meaning
- Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Series (NeurIPS 2025 · similarity 0.93)
- Empowering Time Series Analysis with Large-Scale Multimodal Pretraining (NeurIPS 2026 · similarity 0.92)
- Rethinking Post-Training Recipes for Multimodal Time-Series Forecasting (NeurIPS 2026 · similarity 0.89)
- Adaptive Multi-Scale Decomposition Framework for Time Series Forecasting (AAAI 2025 · similarity 0.88)
- ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data (AAAI 2025 · similarity 0.88)
- TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.