Research map / Time series / Causal and generative time series
An Analysis of Linear Time Series Forecasting Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
The paper proves that DLinear, FITS (without low-pass filter, when L ≥ T−2) and RLinear/NLinear/IN variants reduce to (constrained) linear regression, so closed-form OLS solutions can replace SGD training, and OLS wins in 72% of settings.
Problem
Many recent linear time series forecasting variants (DLinear, FITS, RLinear, NLinear) claim architectural superiority, but their functional expressiveness and training behaviour relative to plain linear regression are not well understood.
Approach
Characterise the model classes of popular linear forecasters analytically: DLinear and FITS equal unconstrained affine maps (FITS when L≥T−2, no LPF), while instance/RevIN and NowNorm impose row-sum or bias-coupling constraints; each can be reformulated as (weakly constrained) linear regression solvable in closed form.
Key results
- Closed-form OLS+IN achieves lower MSE than the corresponding gradient-trained linear model in 23 of 32 (72%) dataset-horizon settings.
- Learned weight matrices of RLinear, DLinear+IN, FITS+IN and OLS+IN are near identical on ETTh1 (T=336), with cosine similarity converging toward the OLS solution during training.
- FITS+IN learns a bias term that differs considerably from the other models and is much smaller in magnitude, despite an equivalent model class.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the closed-form OLS solution instead of SGD-trained linear models | Linear forecasting models · 23 of 32 dataset-horizon settings | MSE: 72% | Helped |
| Add a low-pass filter to FITS (ablation) | FITS · Time series forecasting | performance | Hurt |
| Use FITS training mode 2 with context reconstruction loss instead of mode 1 (ablation) | FITS · Time series forecasting | performance | No clear effect |
| Use FITS instead of OLS | Linear forecasting models · ETTh1 and ETTh2 | MSE | Mixed |
| Use instance-normalised linear models instead of corresponding OLS solutions | Linear forecasting models · Long-term multivariate forecasting | MSE: 72% | Mixed |
Surprising findings
- Closed-form OLS usually outperforms SGD-trained linear models with early stopping, despite having no regularisation.
- FITS performs notably better than OLS on the small hourly ETTh datasets, attributed to an implicit restriction on the bias parameter that reduces overfitting.
Limitations
- Analysis restricted to univariate, channel-independent models; cross-channel information is not modelled.
- FITS analysis excludes the low-pass filter and holds only for L ≥ T−2 (and even L, T).
- Equivalence is functional; differing initialisation and parameterisation can still change early-stopped solutions.
Experimental setup
Datasets: ETTm1 ETTm2 ETTh1 ETTh2 ECL Traffic Weather Exchange
Benchmarks: ETTm1 ETTm2 ETTh1 ETTh2 ECL Traffic Weather Exchange
Metrics: MSE
Baselines: DLinear FITS RLinear NLinear Linear
Builds on: DLinear FITS RLinear NLinear reversible instance normalisation
Closest papers by meaning
- Improving Time Series Forecasting via Instance-aware Post-hoc Revision (NeurIPS 2025 · similarity 0.87)
- AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting (ICML 2025 · similarity 0.87)
- A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle (ICML 2025 · similarity 0.86)
- Unlocking the Power of Patch: Patch-Based MLP for Long-Term Time Series Forecasting (AAAI 2025 · similarity 0.86)
- TimeBase: The Power of Minimalism in Efficient Long-term Time Series Forecasting (ICML 2025 · similarity 0.86)
- AutoTimes: Autoregressive Time Series Forecasters via Large Language Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.