Research map / Time series / Long-term forecasting
Dynamic Relational Priming Improves Transformer in Multivariate Time Series
Hunjae Lee, Corey Clark
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace standard attention with prime attention | Timer-XL · Weather | MSE: 0.277 -> 0.259 | Improved |
| replace standard attention with prime attention | iTransformer · Solar-Energy | MSE: 0.238 -> 0.229 | Improved |
| replace standard attention with prime attention | iTransformer · ECL (homogeneous channels) | MSE: 0.174 -> 0.175 | No clear effect |
| replace standard attention with prime attention | Timer-XL, FreDF, iTransformer · short-term forecasting (PEMS) | MSE: average gain of 5.5% | Improved |
| use prime attention with shorter look-back (L=48 or 64) instead of standard attention at L=96 (ablation) | iTransformer · sequence length analysis, H=96 | MSE: matches standard at L=96 using 40% and 33% less input | Improved |
| add an extra attention head to standard attention (about 38% more parameters) (ablation) | iTransformer standard attention · ETTh1, Weather, Solar | MSE: ETTh1 0.386 -> 0.385; Weather 0.176 -> 0.176; Solar 0.207 -> 0.209 | No clear effect |
| add prime attention instead of extra head (ablation) | iTransformer · ETTh1, Weather, Solar | MSE: ETTh1 0.386 -> 0.378; Weather 0.176 -> 0.170; Solar 0.207 -> 0.199 | Improved |
| use prime attention instead of DIFF Transformer on iTransformer | iTransformer · ETTh2 | MSE: 0.386 -> 0.377 | Improved |
| use prime attention instead of DIFF Transformer | iTransformer · ECL | MSE: 0.173 -> 0.175 | Hurt |
Closest papers by meaning
- Revisiting Attention for Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.87)
- A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle (ICML 2025 · similarity 0.87)
- Introducing Spectral Attention for Long-Range Dependency in Time Series Forecasting (NeurIPS 2024 · similarity 0.86)
- Linear Transformers as VAR Models: Aligning Autoregressive Attention Mechanisms with Autoregressive Forecasting (ICML 2025 · similarity 0.86)
- Sequence Complementor: Complementing Transformers for Time Series Forecasting with Learnable Sequences (AAAI 2025 · similarity 0.86)
- TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
