Research map / Time series / Foundation models for time series
In-Context Fine-Tuning for Time-Series Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Continues pretraining the TimesFM time-series foundation model to accept multiple related in-context time-series examples at inference, yielding TimesFM-ICF with better zero-shot forecasting than its base model and per-dataset fine-tuned variants.
Problem
Zero-shot time-series foundation models for forecasting cannot exploit related example time-series from the target domain at inference; fine-tuning them recovers accuracy but breaks the zero-shot paradigm.
Approach
Continue pretraining a decoder-only TimesFM checkpoint with contexts that pack up to 50 related in-context time-series separated by a learnable separator token, with cross-example causal attention and NoPE positional encoding, so the model adapts to the target distribution at inference without gradient updates.
Key results
- On Monash, TimesFM-ICF reaches scaled MAE (GM) 0.643 versus 0.694 for TimesFM (base) and 0.700 for N-BEATS (about 7% improvement).
- On ETT (horizons 96 and 192), TimesFM-ICF achieves average MAE 0.207/0.265 for horizon 96/192 averaged over datasets, more than 25% better than the nearest baseline.
- TimesFM-ICF slightly beats per-dataset fine-tuned TimesFM (Full) on Monash (0.643 vs 0.663) while total inference takes 4.13 minutes versus 207 minutes for full fine-tuning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add in-context examples at inference time | TimesFM-ICF · Monash | Scaled MAE (GM): 0.694 -> 0.643 | Helped |
| add in-context examples at inference time | TimesFM-ICF · ETT average, horizon 96 | MAE: 0.335 -> 0.207 | Helped |
| add in-context examples at inference time | TimesFM-ICF · ETT average, horizon 192 | MAE: 0.368 -> 0.265 | Helped |
| increase in-context examples from 1 to 50 (ablation) | TimesFM-ICF · ETT, ETTh1 and ETTh2 average | MAE: 0.411 -> 0.345 | Helped |
| increase in-context examples from 1 to 50 (ablation) | TimesFM-ICF · Monash | Scaled MAE (GM): 0.667 -> 0.643 | Helped |
| use 4 in-context examples instead of a 2048-step history (ablation) | TimesFM-ICF-4ex · Monash | Scaled MAE (GM): 0.685 -> 0.675 | Helped |
| use a 2048-step history instead of the standard TimesFM history (ablation) | TimesFM (LH) · Monash | Scaled MAE (GM): 0.694 -> 0.685 | Helped |
| fine-tune all weights per dataset instead of using in-context fine-tuning | TimesFM · Monash | scaled MAE (GM): 0.643 -> 0.663 | Hurt |
| use no positional encodings instead of absolute positional encodings (ablation) | TimesFM (base) · forecasting accuracy | accuracy | No clear effect |
Surprising findings
- In-context fine-tuning without gradient updates outperforms per-dataset full fine-tuning of the base foundation model on Monash, which the authors attribute possibly to catastrophic forgetting on small datasets.
- Performance on Monash improves monotonically with the number of in-context examples up to 50 on ETT datasets.
Limitations
- Only adapts one base foundation model (TimesFM); generality to other time-series foundation models is untested.
- Wiki datasets were excluded from continued pretraining because their series are unrelated and clustering was left for future work.
- Gains are smaller on datasets with short histories and in some cases the model underperforms the base model (e.g., australian electricity demand relative to some baselines and pedestrian counts).
Experimental setup
Datasets: Synthetic Electricity Traffic Weather Favorita Sales LibCity M4 Wiki Trends Monash ETT
Benchmarks: Monash (18 datasets) ETTh1 ETTh2 ETTm1 ETTm2
Metrics: scaled MAE (geometric mean) MAE
Baselines: TimesFM (base) TimesFM (LH) FT-TimesFM (Full) FT-TimesFM (LP) LLMTime N-BEATS PatchTST N-HiTS Informer Pyraformer FEDformer Autoformer
Builds on: TimesFM PatchTST N-BEATS
Closest papers by meaning
- Multi-Scale Finetuning for Encoder-based Time Series Foundation Models (NeurIPS 2025 · similarity 0.91)
- Less is More: Unlocking Specialization of Time Series Foundation Models via Structured Pruning (NeurIPS 2025 · similarity 0.88)
- MOMENT: A Family of Open Time-series Foundation Models (ICML 2024 · similarity 0.88)
- Large Pre-trained time series models for cross-domain Time series analysis tasks (NeurIPS 2024 · similarity 0.87)
- SEMPO: Lightweight Foundation Models for Time Series Forecasting (NeurIPS 2025 · similarity 0.87)
- Learning the Context of Errors: Black-Box Online Adaptation of Time Series Foundation Models (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.