Research map / Time series / Foundation models for time series
AutoTimes: Autoregressive Time Series Forecasters via Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Repurposes frozen decoder-only LLMs (e.g., LLaMA-7B) as autoregressive time series forecasters by embedding segments and LLM-encoded timestamps, and introduces in-context forecasting with time series prompts.
Problem
Prior LLM-based time series forecasters use LLMs non-autoregressively (flattened projection heads, encoder-style), discarding the decoder-only, next-token nature of LLMs and limiting flexible multi-step forecasting and efficiency.
Approach
Freeze the LLM and train only segment embedding and projection MLPs with next-token (segment) prediction, so the forecaster keeps autoregressive rolling generation for arbitrary lengths; use LLM-embedded textual timestamps as position embeddings and feed earlier time series as in-context prompts.
Key results
- On one-for-all long-term forecasting, AutoTimes (LLaMA-7B) averages MSE 0.389 on ETTh1 and 0.159 on ECL, beating TimeLLM and other baselines in 80% of datasets; it is reported to outperform TimeLLM by 9.12% on average.
- Short-term M4 average sMAPE 11.831 vs 11.983 for TimeLLM, with the best OWA of 0.850.
- Trains only about 0.1% of parameters (0.79M MLP) with the LLM frozen, and reports over 5x training/inference speedup versus LLM4TS baselines; in-context forecasting reduces zero-shot M4->M3 SMAPE by 13.3% on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AutoTimes instead of Koopa | M4 short-term forecasting · M4 average | sMAPE: 11.863 -> 11.831 | Helped |
| use AutoTimes instead of TimeLLM | one-for-all forecasting · ETTh1 | MSE: 0.412 -> 0.389 | Helped |
| use AutoTimes instead of TimeLLM | one-for-all forecasting · Weather | MSE: 0.225 -> 0.235 | Hurt |
| use AutoTimes instead of FPT | zero-shot forecasting · M4 → M3 | SMAPE: 13.06 -> 12.75 | Helped |
| add target-domain time-series prompts | AutoTimes · M4 → M3 in-context forecasting | SMAPE: 13.3% SMAPE reduction | Helped |
| replace the LLM with direct input tokens (ablation) | AutoTimes · ETTh1, Pred-96 | MSE: 0.360 -> 0.365 | Hurt |
| replace the LLM with direct input tokens (ablation) | AutoTimes · ECL, Pred-96 | MSE: 0.129 -> 0.171 | Hurt |
| replace the LLM with one multi-head attention layer (ablation) | AutoTimes · ETTh1, Pred-96 | MSE: 0.360 -> 0.383 | Hurt |
| replace the LLM with one transformer block (ablation) | AutoTimes · ETTh1, Pred-96 | MSE: 0.360 -> 0.377 | Hurt |
| add LoRA to intermediate LLM layers (ablation) | AutoTimes · ECL, Pred-96 | MSE: 0.140 -> 0.130 | Helped |
| add LoRA to intermediate LLM layers (ablation) | AutoTimes · ETTh1, Pred-336 | MSE: 0.408 -> 0.409 | Hurt |
| add LLM-embedded textual timestamp position embeddings | AutoTimes · time-series forecasting | forecasting performance | Helped |
Surprising findings
- Non-autoregressive LLM4TS methods degrade markedly without per-horizon retraining, while deep forecasters like DLinear and PatchTST remain competitive under rolling forecasting.
- Timestamps embedded via frozen LLM at no extra context length improve forecasting, even though prior deep forecasters found timestamp embeddings had little effect.
Limitations
- Relies on frozen intermediate LLM layers; LoRA gives only modest gains and was not the default.
- Evaluation uses the authors' one-for-all rolling protocol and reproduced baselines, which may differ from official reports.
- Future work limited to newer low-rank adaptation and stronger language backbones.
Experimental setup
Datasets: ETTh1 ECL Traffic Weather Solar-Energy M4 M3
Benchmarks: ETTh1 ECL Traffic Weather Solar-Energy M4 M3
Metrics: MSE MAE sMAPE MASE OWA
Baselines: TimeLLM UniTime FPT iTransformer DLinear PatchTST TimesNet Koopa N-HiTS N-BEATS FiLM LLMTime
Builds on: TimesNet benchmark FPT Time-LLM LoRA channel independence
Closest papers by meaning
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.93)
- Are Language Models Actually Useful for Time Series Forecasting? (NeurIPS 2024 · similarity 0.91)
- LangTime: A Language-Guided Unified Model for Time Series Forecasting with Proximal Policy Optimization (ICML 2025 · similarity 0.90)
- $S^2$IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series Forecasting (ICML 2024 · similarity 0.89)
- TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment (AAAI 2025 · similarity 0.89)
- Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.