Research map / Time series / Causal and generative time series
Scaling Law for Time Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an intrinsic-space scaling-law theory for time series forecasting that accounts for look-back horizon, dataset size and model size, and validates it empirically on several datasets with linear, MLP, ModernTCN and iTransformer models.
Problem
Deep time series forecasters show abnormal scaling: longer look-back horizons can hurt performance, and more capable models do not always beat simpler ones. Existing scaling-law theories do not account for the look-back horizon.
Approach
Models forecasting in an intrinsic space whose dimension grows with the horizon; the loss decomposes into a Bayesian term that falls with horizon and an approximation term (piecewise-linear model with dataset-noise and nearest-neighbor regimes) that rises with horizon, yielding an optimal horizon that grows with dataset size and shrinks with model size.
Key results
- Optimal look-back horizon exists for each data amount and grows as training data increases (e.g., ETTh1 optimal horizon >300 with 11% data vs. <30 for Exchange with 100%).
- Data-scaling loss fits C + 1/D^alpha and width-scaling loss fits C + 1/W^alpha well across models and datasets, except ModernTCN on ETTm1 where large models overfit.
- Optimal horizon is largely insensitive to model size when the model is small, and decreases with larger model size in the large-model regime.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use different starting slices of the training set while keeping 10% of training data available | NLinear · Weather | MSE: $0.222\pm 0.001$; $0.221\pm 0.001$; $0.223\pm 0.001$; $0.223\pm 0.001$ | No clear effect |
| Increase dataset size | time series forecasting models · ETTh1, ETTh2, ETTm1, ETTm2, Traffic, Weather, and other evaluated datasets | loss | Helped |
| Increase model width when the model is not powerful enough | time series forecasting models · Evaluated time series forecasting datasets | loss | Helped |
| Increase model size when the model already approximates the data | ModernTCN · ETTm1 | loss | Hurt |
| Increase look-back horizon beyond the dataset-specific optimum | time series forecasting models · Evaluated forecasting datasets | loss | Hurt |
| Increase available training data | time series forecasting models · Evaluated forecasting datasets | optimal look-back horizon | Helped |
| Use a dataset with stronger feature degradation | Linear model · Exchange versus ETTh1 | optimal look-back horizon: Exchange: $<30$; ETTh1: $>300$ | Hurt |
| Increase model size when model size is already optimal | ModernTCN · ETTh1 and Weather | optimal look-back horizon | Hurt |
Surprising findings
- Longer look-back horizons can hurt performance even though more historical information is available, because the optimal horizon is bounded by dataset size.
- Larger models can fail to improve and may hurt performance on small data (ModernTCN on ETTm1).
Limitations
- Experiments use only small and medium datasets; large foundation datasets are not tested.
- Theory relies on several assumptions (intrinsic space, Lipschitz, Zipf eigenvalue decay) that are not strictly verified and only in-domain settings are covered.
- Only a few model families (NLinear/MLP, ModernTCN, iTransformer) and a fixed prediction length of 192 are studied.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Exchange Weather ECL Traffic
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Exchange Weather ECL Traffic
Metrics: MSE loss vs. dataset size fit loss vs. look-back horizon
Baselines: NLinear ModernTCN iTransformer
Builds on: NLinear ModernTCN iTransformer PatchTST
Closest papers by meaning
- Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts (ICLR 2025 · similarity 0.89)
- SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Series (NeurIPS 2025 · similarity 0.88)
- Scaling Laws in Linear Regression: Compute, Parameters, and Data (NeurIPS 2024 · similarity 0.88)
- On the Invariance and Generality of Neural Scaling Laws (NeurIPS 2026 · similarity 0.87)
- A Dynamical Model of Neural Scaling Laws (ICML 2024 · similarity 0.87)
- TimeBase: The Power of Minimalism in Efficient Long-term Time Series Forecasting (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.