Research map / Time series / Foundation models for time series
In-context Time Series Predictor
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ICTSP, a Transformer that treats (lookback, future) forecasting pairs built from the series as input tokens so that time series forecasting uses in-context learning without pretrained LLM weights.
Problem
Existing Transformer TSF models suffer from timestep mixing, permutation invariance, overfitting, and fixed channel structure, and LLM-based TSF adapts large pretrained models inefficiently; the paper asks how to use in-context learning directly for time series forecasting.
Approach
Reformulate forecasting tasks as tokens: each token is a ground-truth (lookback, future) pair sampled from the series, with the target's lookback as a query token, so a small Transformer learns a predictor in context without parameter updates or pretrained LLMs.
Key results
- Full-data: ICTSP average MSE is best on 18 of 28 dataset-horizon tasks with AvgRank 1.36 (e.g., ECL avg MSE 0.154 vs Time-LLM 0.159).
- Zero-shot ETT transfer: ICTSP has AvgRank 1.13 and is best in 21 of 24 cases (e.g., ETTh1→ETTh2 MSE 0.337 vs Time-LLM 0.352).
- Few-shot 10% training: ICTSP achieves AvgRank 1.57 and best average MSE on ETTh1 (0.525) versus Time-LLM 0.555.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Reformulate forecasting tasks (lookback, future pairs) as input tokens with ICL context (ICTSP) | Transformer for TSF · Full-data TSF, 7 datasets | AvgRank: AvgRank 1.36 vs 1.93 for next best (Time-LLM) | Helped |
| Context example tokens removed (reduces to series-wise Transformer) (ablation) | ICTSP · Ablation on Weather and ETTm2 | MSE avg: Weather 0.218 -> 0.225; ETTm2 0.247 -> 0.251 | Hurt |
| Token retrieval removed (ablation) | ICTSP · Ablation on Weather and ETTm2 | MSE avg: Weather 0.218 -> 0.219; ETTm2 0.247 -> 0.246 | No clear effect |
| Sampling step increased from m=8 to m=256 (ablation) | ICTSP · Ablation on Weather and ETTm2 | MSE avg: Weather 0.218 -> 0.220; ETTm2 0.247 -> 0.250 | Hurt |
| Sampling step m=1 (denser context) vs m=8 (ablation) | ICTSP · Ablation on Weather and ETTm2 | MSE avg: Weather 0.218 -> 0.218; ETTm2 0.247 -> 0.246 | No clear effect |
| Train on only 10% of training data (few-shot) | ICTSP · Few-shot 10% TSF, 7 datasets | AvgRank: AvgRank 1.57 vs 1.89 for Time-LLM | Helped |
| Zero-shot transfer across ETT datasets | ICTSP · Zero-shot ETT transfer, 6 pairs | AvgRank: AvgRank 1.13 vs 1.88 for Time-LLM | Helped |
Surprising findings
- Removing context examples (reducing ICTSP to a Series-wise Transformer) significantly degrades performance, indicating the in-context examples drive the gain.
- Increasing the sampling step m to 256 degrades performance toward the no-context level, while m=8 with token retrieval loses little versus m=1.
Limitations
- Scaling to more layers and larger datasets was not tested due to the lack of large collective TSF datasets.
- Training a single global model that handles different lookback and future lengths was not explored.
- Few-shot evaluation with the 5% setting lacks data for the 720 horizon.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity (ECL) Traffic
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Weather ECL Traffic
Metrics: MSE MAE average rank number of rank-1 results FLOPs parameter count
Baselines: Time-LLM GPT4TS LLMTime PatchTST FEDformer Autoformer Informer iTransformer TimesNet DLinear
Builds on: PatchTST iTransformer ForecastPFN
Closest papers by meaning
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.91)
- Are Language Models Actually Useful for Time Series Forecasting? (NeurIPS 2024 · similarity 0.90)
- AutoTimes: Autoregressive Time Series Forecasters via Large Language Models (NeurIPS 2024 · similarity 0.89)
- TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents (AAAI 2025 · similarity 0.88)
- Timer: Generative Pre-trained Transformers Are Large Time Series Models (ICML 2024 · similarity 0.88)
- Timer-XL: Long-Context Transformers for Unified Time Series Forecasting (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.