Research map / Time series / Long-term forecasting
Considering Nonstationary within Multivariate Time Series with Variational Hierarchical Transformer for Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HTV-Trans, a hierarchical variational generative module combined with a transformer that restores non-stationary information lost by series stationarization for multivariate time series forecasting.
Problem
Multivariate time series forecasting suffers from non-stationarity; stationarization methods cause over-stationarization and deterministic models ignore stochasticity of MTS.
Approach
A hierarchical multi-scale probabilistic generative module (HTPGM) with VAE-style inference is coupled with a transformer; its latent summaries are fused with the normalized input to recover intrinsic non-stationarity, trained with a combined prediction and reconstruction ELBO.
Key results
- HTV-Trans achieves best MSE and MAE in most settings on ETTh2, ETTm2 and Weather, e.g., ETTh2 horizon 96 MSE 0.300 vs 0.346 for FEDformer.
- Ablation on ETTh2/Weather/Exchange shows combined prediction+reconstruction objective with HTPGM lowers MSE (e.g., ETTh2-96 from 0.368 to 0.338 relative to plain Transformer).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HTV-Trans instead of the strongest listed baseline | ETTh2 forecasting · ETTh2, prediction length 96 | MSE: 0.346 -> 0.300 | Helped |
| use HTV-Trans instead of the strongest listed baseline | ETTh2 forecasting · ETTh2, prediction length 96 | MAE: 0.388 -> 0.338 | Helped |
| use HTV-Trans instead of the strongest listed baseline | Weather forecasting · Weather, prediction length 192 | MSE: 0.219 -> 0.216 | Helped |
| use HTV-Trans instead of the strongest listed baseline | Weather forecasting · Weather, prediction length 192 | MAE: 0.285 -> 0.257 | Helped |
| use HTV-Trans instead of the strongest listed baseline | Exchange forecasting · Exchange, prediction length 96 | MSE: 0.111 -> 0.086 | Helped |
| use HTV-Trans instead of the strongest listed baseline | Exchange forecasting · Exchange, prediction length 96 | MAE: 0.237 -> 0.205 | Helped |
| add HTPGM and use the combined prediction and reconstruction objective (ablation) | Transformer forecasting · ETTh2, prediction length 96 | MAE: 0.368 -> 0.338 | Helped |
| add HTPGM and use the combined prediction and reconstruction objective (ablation) | Transformer forecasting · Weather, prediction length 96 | MAE: 0.230 -> 0.223 | Helped |
| add HTPGM and use the combined prediction and reconstruction objective (ablation) | Transformer forecasting · Exchange, prediction length 96 | MAE: 0.246 -> 0.205 | Helped |
| replace traditional HVAE with HTPGM under the combined objective (ablation) | Transformer forecasting · ETTh2, prediction length 96 | MAE: 0.362 -> 0.338 | Helped |
| replace traditional HVAE with HTPGM under the combined objective (ablation) | Transformer forecasting · Weather, prediction length 96 | MAE: 0.228 -> 0.223 | Helped |
| replace traditional HVAE with HTPGM under the combined objective (ablation) | Transformer forecasting · Exchange, prediction length 96 | MAE: 0.241 -> 0.205 | Helped |
Surprising findings
- On ETTh1, FEDformer and Crossformer obtain lower MSE than HTV-Trans at some horizons, so the method is not uniformly best.
Limitations
- Performance not uniformly best; FEDformer and Crossformer win some ETTh1 and Weather settings.
- Hyperparameters alpha and gamma chosen by grid search; sensitivity relies on ablation.
- Benchmark results partly quoted from original papers, so comparability may vary.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Weather ILI Exchange
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Weather ILI Exchange
Metrics: MSE MAE
Baselines: Autoformer Informer NS Transformer FEDformer Pyraformer Crossformer
Builds on: Non-stationary Transformer RevIN Autoformer Informer
Closest papers by meaning
- HDT: Hierarchical Discrete Transformer for Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.91)
- Latent Diffusion Transformer for Probabilistic Time Series Forecasting (AAAI 2024 · similarity 0.88)
- SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention (ICML 2024 · similarity 0.88)
- $K^2$VAE: A Koopman-Kalman Enhanced Variational AutoEncoder for Probabilistic Time Series Forecasting (ICML 2025 · similarity 0.87)
- TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data (AAAI 2025 · similarity 0.87)
- Sequence Complementor: Complementing Transformers for Time Series Forecasting with Learnable Sequences (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.