Research map / Time series / Foundation models for time series
Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Time-LLM reprograms a frozen LLM (mainly Llama-7B) for time series forecasting by mapping patch embeddings onto learned text prototypes and prepending natural-language prompts (Prompt-as-Prefix), training only lightweight input and output layers.
Problem
Adapting large language models to general time series forecasting, which requires aligning continuous time series with discrete language tokens without costly fine-tuning of the backbone, and achieving accuracy in standard, few-shot and zero-shot settings.
Approach
Keep the LLM frozen and reprogram time series patches via cross-attention onto a small set of learned text prototypes derived from the word embedding matrix, then prepend a natural-language prompt (dataset context, task instruction, input statistics such as trends and lags) as a prefix; only the input embedding and output projection are trained.
Key results
- Long-term forecasting: average MSE 0.408 on ETTh1 and 36 SOTA instances out of 40 across eight benchmarks; 12% average gain over GPT4TS and 20% over TimesNet.
- Few-shot (10% data): about 5% MSE reduction vs GPT4TS; zero-shot ETT cross-domain: over 14.2% MSE reduction vs second-best and 22% vs GPT4TS.
- M4 short-term: OWA 0.859 vs GPT4TS 0.94, outperforming GPT4TS by 8.7%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reprogram a frozen Llama-7B with text prototypes and Prompt-as-Prefix | Time-LLM · ETTm1 | MSE | Helped |
| use Time-LLM rather than PatchTST | long-term forecasting · ETTh2 | MSE: 0.330 -> 0.334 | Hurt |
| use Time-LLM rather than PatchTST | long-term forecasting · Weather | MSE: 0.225 -> 0.225 | No clear effect |
| use Time-LLM rather than GPT4TS | M4 short-term forecasting · M4 | SMAPE: +8.7% | Helped |
| use the full Llama (32) rather than Llama (8) (ablation) | Time-LLM · ETTh1 and ETTm1 forecasting | MSE: 14.5% | Helped |
| use GPT-2 (12) rather than GPT-2 (6) (ablation) | Time-LLM · ETTh1 and ETTm1 forecasting | MSE: 2.7% | Helped |
| add patch reprogramming (ablation) | Time-LLM · standard and few-shot forecasting | MSE: 9.2%; exceeding 17% in few-shot tasks | Helped |
| add Prompt-as-Prefix (ablation) | Time-LLM · standard and few-shot forecasting | MSE: over 8%; over 19% in few-shot forecasting | Helped |
| add input statistics (ablation) | Time-LLM prompts · forecasting | MSE: 10.2% | Helped |
| add dataset context (ablation) | Time-LLM prompts · forecasting | MSE: 9.6% | Helped |
| add task instruction (ablation) | Time-LLM prompts · forecasting | MSE: 7.7% | Helped |
Surprising findings
- Removing input statistics (prompt context) hurts most, increasing MSE by about 10.2% on average, showing prompt content matters.
- Llama-7B with full 32 layers outperforms its 8-layer variant by 14.5%, and the scaling law holds after reprogramming.
Limitations
- Relies on embedding-visible LLMs such as Llama and GPT-2; not applicable to closed models without embedding access.
- Overall efficiency is capped by the backbone size (Llama-32 layers has about 3.4B parameters including the backbone per the efficiency table, and high memory use).
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity Traffic ILI M4 M3-Quarterly
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity Traffic ILI M4 M3-Quarterly
Metrics: MSE MAE SMAPE MASE OWA
Baselines: GPT4TS DLinear PatchTST TimesNet FEDformer Autoformer Stationary Transformer ETSformer LightTS Informer Reformer N-HiTS
Builds on: PatchTST reversible instance normalization Voice2Series GPT4TS
Closest papers by meaning
- AutoTimes: Autoregressive Time Series Forecasters via Large Language Models (NeurIPS 2024 · similarity 0.93)
- Are Language Models Actually Useful for Time Series Forecasting? (NeurIPS 2024 · similarity 0.92)
- TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment (AAAI 2025 · similarity 0.91)
- $S^2$IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series Forecasting (ICML 2024 · similarity 0.91)
- Can LLMs Understand Time Series Anomalies? (ICLR 2025 · similarity 0.91)
- TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents (AAAI 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.