Research map / Time series / Foundation models for time series
Multi-Scale Finetuning for Encoder-based Time Series Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MSFT, a causal-motivated multi-scale finetuning framework for encoder-based time series foundation models that adds scale-specific adapters, LoRA, in-scale masking, cross-scale aggregators, and learnable scale mixing, outperforming naive finetuning and PEFT baselines.
Problem
Naive finetuning of time series foundation models (TSFMs) overfits to the original sampling scale and underuses their multi-scale forecasting capabilities; effective finetuning of encoder-based TSFMs for downstream forecasting remains underexplored.
Approach
Views scale as a confounder and approximates the backdoor adjustment P(Y|do(X)) by building downsampled multi-scale series, activating scale-specific knowledge via scale-specific adapters and LoRA, decoupling in-scale attention from cross-scale aggregation, and mixing per-scale predictions with learnable softmax weights.
Key results
- On Moirai-Base, MSFT reaches 0.332 MSE on ETTm1 versus 0.368 for full finetuning and 0.359 for AdaLoRA, the best finetuned result.
- MSFT achieves 24.4% CRPS relative reduction on Solar versus full finetuning on Moirai-Base (0.142 vs 0.188).
- Ablations show removing cross-scale aggregators or the in-scale mask degrades LSF MSE, e.g., ETTm1 0.354 with full MSFT vs 0.357-0.371 for ablations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MSFT instead of full finetuning | Moirai Small · ETTm1 (LSF; averaged over prediction lengths) | MSE: 0.367 -> 0.353 | Helped |
| use MSFT instead of full finetuning | Moirai Small · ETTm2 (LSF; averaged over prediction lengths) | MSE: 0.273 -> 0.250 | Helped |
| use MSFT instead of full finetuning | Moirai Small · Weather (LSF; averaged over prediction lengths) | MSE: 0.228 -> 0.216 | Helped |
| use MSFT instead of full finetuning | Moirai Small · Solar (probabilistic forecasting) | CRPS: 0.395 -> 0.353 | Helped |
| add scale-specific attention modules rather than freeze shared attention (ablation) | MSFT on Moirai Small · ETTm1 (LSF; ablation) | MSE: 0.362 -> 0.354 | Helped |
| add scale-specific attention modules rather than shared finetuned attention (ablation) | MSFT on Moirai Small · ETTm1 (LSF; ablation) | MSE: 0.360 -> 0.354 | Helped |
| add scale-specific input adapters rather than freeze the shared input projection (ablation) | MSFT on Moirai Small · ETTm1 (LSF; ablation) | MSE: 0.374 -> 0.354 | Helped |
| add scale-specific input adapters rather than use a shared finetuned input projection (ablation) | MSFT on Moirai Small · ETTm1 (LSF; ablation) | MSE: 0.361 -> 0.354 | Helped |
| add coarse-to-fine and fine-to-coarse cross-scale aggregators (ablation) | MSFT on Moirai Small · ETTm1 (LSF; ablation) | MSE: 0.371 -> 0.354 | Helped |
| add the coarse-to-fine aggregation branch (ablation) | MSFT on Moirai Small · ETTm1 (LSF; ablation) | MSE: 0.363 -> 0.354 | Helped |
| add the fine-to-coarse aggregation branch (ablation) | MSFT on Moirai Small · ETTm1 (LSF; ablation) | MSE: 0.357 -> 0.354 | Helped |
| use decoupled dependency modeling instead of direct attention over concatenated multi-scale tokens (ablation) | MSFT on Moirai Small · ETTm1 (LSF; ablation) | MSE: 0.360 -> 0.354 | Helped |
Surprising findings
- Naive full finetuning of Moirai can be worse than zero-shot on some datasets, and freezing the input projection (instead of finetuning shared weights) causes larger drops in ablations than sharing weights.
Limitations
- Moment is evaluated only in its Small size due to computational constraints, and gains are less pronounced on Moment.
- Applicability is restricted to encoder-based TSFMs; decoder-based extension is only discussed in the appendix.
- Learning rates and configurations are selected from small grids based on validation performance.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity Solar Istanbul Traffic Turkey Power Jena Weather
Benchmarks: ETTm1 ETTm2 ETTh1 ETTh2 Electricity Weather Solar Istanbul Traffic Turkey Power
Metrics: MSE MAE CRPS MSIS
Baselines: Full finetuning Linear probing Prompt tuning LoRA AdaLoRA DLinear PatchTST iTransformer TimeMixer SimpleTM DeepAR TFT
Builds on: LoRA AdaLoRA Moirai Moment UniTS
Closest papers by meaning
- Less is More: Unlocking Specialization of Time Series Foundation Models via Structured Pruning (NeurIPS 2025 · similarity 0.91)
- In-Context Fine-Tuning for Time-Series Foundation Models (ICML 2025 · similarity 0.91)
- Towards Neural Scaling Laws for Time Series Foundation Models (ICLR 2025 · similarity 0.90)
- Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer (ICML 2025 · similarity 0.88)
- TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecaster (NeurIPS 2025 · similarity 0.88)
- Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.