Research map / Time series / Long-term forecasting
Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Leddam, a multivariate forecasting model with a learnable Gaussian-initialized convolutional trend-seasonal decomposition and a dual attention module (channel-wise plus auto-regressive self-attention), evaluated on eight datasets against iTransformer, PatchTST, and others.
Problem
Multivariate time series forecasting must capture inter-series dependencies and intra-series variations, but fixed moving-average trend decomposition is non-adaptive and point/patch attention loses information or is poorly suited to temporal order.
Approach
Replace the fixed moving-average kernel with a trainable 1D convolution initialized with Gaussian (softmax-normalized) weights for trend-seasonal decomposition, and model the seasonal part with two parallel transformer branches: channel-wise attention over whole-series tokens and auto-regressive (cyclically shifted) token attention for temporal variations.
Key results
- Leddam achieves best average MSE on 7 of 8 datasets with lookback 96 (e.g., ETTh1 avg MSE 0.431 vs iTransformer 0.454), with iTransformer best on Traffic.
- Plugging the learnable decomposition into LightTS, LSTM, SCINet, Informer, and Transformer reduces MSE by 11.87% to 48.56% on average across five datasets.
- Trainable learnable decomposition gives an average 11.98% MSE decrease over the moving average kernel in the DLinear substitution experiment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Leddam instead of the strongest baseline | multivariate forecasting models · Traffic (average, F=96/192/336/720) | MSE: 0.428 -> 0.467 | Hurt |
| Use Leddam instead of the strongest baseline | multivariate forecasting models · ETTh1 (average, F=96/192/336/720) | MSE: 0.434 -> 0.431 | Helped |
| Use Leddam instead of the strongest baseline | multivariate forecasting models · ETTh2 (average, F=96/192/336/720) | MSE: 0.383 -> 0.373 | Helped |
| Use Leddam instead of the strongest baseline | multivariate forecasting models · ETTm1 (average, F=96/192/336/720) | MSE: 0.387 -> 0.386 | Helped |
| Use Leddam instead of the strongest baseline | multivariate forecasting models · ETTm2 (average, F=96/192/336/720) | MSE: 0.283 -> 0.281 | Helped |
| Use Leddam instead of the strongest baseline | multivariate forecasting models · Electricity (average, F=96/192/336/720) | MSE: 0.178 -> 0.169 | Helped |
| Use Leddam instead of the strongest baseline | multivariate forecasting models · Solar-Energy (average, F=96/192/336/720) | MSE: 0.233 -> 0.230 | Helped |
| Use Leddam instead of the strongest baseline | multivariate forecasting models · Weather (average, F=96/192/336/720) | MSE: 0.243 -> 0.242 | Helped |
| add autoregressive self-attention (ablation) | Leddam · ETTm1, Electricity, Traffic, Weather, Solar-Energy (F=96) | MSE: 0.337 -> 0.320; 0.148 -> 0.139; 0.438 -> 0.424; 0.168 -> 0.158; 0.211 -> 0.202 | Helped |
| add channel-wise self-attention (ablation) | Leddam · ETTm1, Electricity, Traffic, Weather, Solar-Energy (F=96) | MSE: 0.335 -> 0.320; 0.152 -> 0.139; 0.467 -> 0.424; 0.167 -> 0.158; 0.226 -> 0.202 | Helped |
| add both attention branches instead of replacing them with a linear layer (ablation) | Leddam · ETTm1, Electricity, Traffic, Weather, Solar-Energy (F=96) | MSE: 0.350 -> 0.320; 0.197 -> 0.139; 0.644 -> 0.424; 0.195 -> 0.158; 0.306 -> 0.202 | Helped |
Surprising findings
- Adding the learnable decomposition to LSTM cut MSE by 76.97% on ETTh2 and 80.28% on ETTm2, far larger than gains for the other backbones.
- Several baseline numbers in the main table are reported with a fixed bug fix noted from FITS, and the decomposition module's kernel size has little effect on results (k from 15 to 105 changes MSE by under 0.001 on ETTm1).
Limitations
- Traffic is the one dataset where Leddam does not give the best result (iTransformer is better).
- Hyperparameters such as dim and number of layers are sensitive, requiring grid search.
- Decomposition quality is evaluated by DTW and FFT similarity proxies rather than by downstream-independent ground truth.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Traffic Solar-Energy Weather
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Traffic Solar-Energy Weather
Metrics: MSE MAE DTW FFT amplitude similarity
Baselines: iTransformer TimesNet MICN DLinear PatchTST Crossformer TiDE SCINet
Builds on: iTransformer DLinear Autoformer RevIN
Closest papers by meaning
- MSGNet: Learning Multi-Scale Inter-series Correlations for Multivariate Time Series Forecasting (AAAI 2024 · similarity 0.88)
- TimeEmb: A Lightweight Static-Dynamic Disentanglement Framework for Time Series Forecasting (NeurIPS 2025 · similarity 0.88)
- OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain (NeurIPS 2025 · similarity 0.88)
- Multivariate Time Series Forecasting needs Cross Variable Loss (NeurIPS 2026 · similarity 0.88)
- Revisiting Attention for Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.87)
- Rethinking Channel Dependence for Multivariate Time Series Forecasting: Learning from Leading Indicators (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.