Research map / Time series / Foundation models for time series
One Fits All: Power General Time Series Analysis by Pretrained LM
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use pretrained GPT2 weights instead of random init (ablation) | GPT2(6) FPT · ETTh1 | MSE: 1.326 -> 0.427 | Helped |
| freeze self-attention and FFN during fine-tuning (ablation) | GPT2(6) FPT · ETTh1 | MSE: 0.753 -> 0.427 | Helped |
| add 6 pretrained GPT2 layers vs none (ablation) | FPT · ETTh2 | MSE: 0.666 -> 0.346 | Helped |
| frozen pretrained GPT2 backbone | time series analysis · Few-shot 10% | MSE: 0.385 -> 0.371 vs PatchTST | Helped |
| frozen pretrained GPT2 backbone | time series analysis · Long-term forecasting | MSE: 0.446 vs 0.516 (PatchTST better) | Hurt |
| frozen pretrained GPT2 backbone | time series analysis · Anomaly detection | F1: 85.24 -> 86.72 | Helped |
| frozen pretrained GPT2 backbone | time series analysis · Zero-shot forecasting | aggregated error: 17.67 -> 16.38 vs PatchTST | Helped |
| use BERT or BEiT backbone instead of GPT2 (ablation) | FPT · ETTh2 5% | MSE: 0.400 -> 0.452 (BERT) | Hurt |
Closest papers by meaning
- Timer: Generative Pre-trained Transformers Are Large Time Series Models (ICML 2024 · similarity 0.90)
- TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting (ICLR 2024 · similarity 0.89)
- Large Pre-trained time series models for cross-domain Time series analysis tasks (NeurIPS 2024 · similarity 0.89)
- A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle (ICML 2025 · similarity 0.87)
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.87)
- MOMENT: A Family of Open Time-series Foundation Models (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.