Research map / Time series / Long-term forecasting
OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OLinear, a linear-based multivariate forecaster that decorrelates series with a data-adaptive orthogonal transform (OrthoTrans, from the temporal Pearson correlation matrix) and uses a NormLin layer (softplus + row-L1-normalized weights) in place of self-attention.
Problem
Temporal forecasting models that encode and decode in the time domain struggle with entangled step-wise dependencies, and fixed transform bases (Fourier, wavelet) ignore dataset-specific correlation structure; self-attention is also costly for multivariate dependency modeling.
Approach
Eigen-decompose the dataset's temporal Pearson correlation matrix to get a data-adaptive orthogonal basis that decorrelates the series before linear encoding/decoding, and replace attention across variates with NormLin, a linear layer whose weights are made positive by Softplus and row-L1-normalized so each row sums to 1 like attention.
Key results
- OLinear achieves the best average results on long-term forecasting, with 5 first-place MSE and 9 first-place MAE counts across the reported long-term benchmark averages (Table 1).
- Replacing self-attention with NormLin improves iTransformer and PatchTST average MSE by 6.7% and 10.3% respectively, while NormLin needs nearly half the FLOPs of multi-head self-attention.
- Applying OrthoTrans as a plug-in yields average MSE improvements of 5.1% for iTransformer and 10.1% for PatchTST; OrthoTrans gives a 5.5% MSE reduction over the Fourier basis on PEMS03.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use dataset-adaptive OrthoTrans instead of the Fourier basis (ablation) | OLinear · PEMS03 | MSE: 0.101 -> 0.095 | Helped |
| add temporal orthogonal transformation instead of using Identity (ablation) | OLinear · PEMS03 | MSE: 0.106 -> 0.095 | Helped |
| replace OrthoTrans with fixed polynomial or wavelet bases (ablation) | OLinear · PEMS03 and other listed datasets | MSE: 0.095 vs. 0.101 (Fourier); 0.106 (Identity) | Hurt |
| add OrthoTrans (ablation) | iTransformer, PatchTST, and RLinear · ETTm1, ECL, PEMS03, PEMS07, Solar, Weather, METR-LA | MSE: iTransformer: 0.407 -> 0.404 (ETTm1); PatchTST: 0.387 -> 0.384 (ETTm1) | Helped |
| replace variate-wise NormLin with a standard linear layer (ablation) | OLinear · ECL and other listed datasets | MSE: 0.159 -> 0.178 (ECL) | Hurt |
| replace variate-wise NormLin with self-attention (ablation) | OLinear · ECL and other listed datasets | MSE: 0.159 -> 0.166 (ECL) | Hurt |
| remove the ISL module (ablation) | OLinear · ECL and other listed datasets | MSE: 0.159 -> 0.169 (ECL) | Hurt |
| remove the NormLin module (ablation) | OLinear · NASDAQ (S1) | MSE: 0.055 -> 0.054 | Helped |
| replace learned NormLin weights with Softmax of the variate correlation matrix (ablation) | OLinear · ECL and short-term datasets | MSE: 0.159 -> 0.161 (ECL); 1.429 -> 1.463 (ILI) | Mixed |
| replace self-attention with NormLin (ablation) | Transformer forecasters · ETTm1, ECL, PEMS03, PEMS07, Solar, Weather, METR-LA | MSE: iTransformer: 0.407 -> 0.388 (ETTm1); PatchTST: 0.387 -> 0.379 (ETTm1) | Helped |
| replace self-attention with NormLin (ablation) | Timer · ETTh2 zero-shot | MSE: 0.314 -> 0.274 | Helped |
| replace self-attention with NormLin (ablation) | Timer · Traffic zero-shot | MSE: 0.458 -> 0.462 | Hurt |
Surprising findings
- NormLin along the variate dimension outperforms standard multi-head self-attention while requiring roughly half the FLOPs.
- On small-variate datasets such as NASDAQ and ILI, a model with only temporal linear layers is competitive, suggesting NormLin helps mainly when the number of variates is large.
Limitations
- Orthogonal matrices Q_i and Q_o are precomputed from training data, so the basis is fixed after computation and dataset-specific.
- NormLin benefits appear mainly when the number of variates is large; small-variate datasets show little gain from the variate-mixing layer.
- The paper leaves further application of NormLin to large time series models and broader time series analysis tasks to future work.
Experimental setup
Datasets: ETT Weather ECL Traffic Exchange Solar-Energy PEMS03 PEMS04 PEMS07 PEMS08 ILI COVID-19
Benchmarks: ETT Weather ECL Traffic Exchange Solar-Energy PEMS03 PEMS04 PEMS07 PEMS08 ILI COVID-19
Metrics: MSE MAE FLOPs GPU memory
Baselines: TimeMixer TimeMixer++ FilterNet FITS DLinear Leddam CARD Fredformer iTransformer PatchTST TimesNet RLinear
Builds on: RevIN CARD iTransformer PatchTST Timer
Closest papers by meaning
- iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (ICLR 2024 · similarity 0.89)
- TimeEmb: A Lightweight Static-Dynamic Disentanglement Framework for Time Series Forecasting (NeurIPS 2025 · similarity 0.89)
- TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting (NeurIPS 2025 · similarity 0.88)
- TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables (NeurIPS 2024 · similarity 0.88)
- Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling (ICML 2024 · similarity 0.88)
- Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.