Research map / Time series / Long-term forecasting
Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use convolutional queries and keys with a wider local context (ablation) | Transformer · traffic-c (1d rolling) | R0.5: 0.134 -> 0.122 | Helped |
| Use convolutional queries and keys with a wider local context (ablation) | Transformer · electricity-c (1d rolling) | R0.5: 0.060 -> 0.057; 0.060 -> 0.059 | No clear effect |
| Replace full attention with LogSparse attention under the same memory budget (ablation) | Transformer · traffic-f (1d rolling) | R0.5: 0.161 -> 0.150 | Helped |
| Replace full attention with LogSparse attention under the same memory budget (ablation) | Transformer · electricity-f (1d rolling) | R0.5: 0.083 -> 0.084 | Hurt |
| Add convolutional attention (k=6) (ablation) | Full-attention Transformer · traffic-f (1d rolling, same memory budget) | R0.5: 0.161 -> 0.149 | Helped |
| Add convolutional attention (k=6) (ablation) | LogSparse Transformer · traffic-f (1d rolling, same memory budget) | R0.5: 0.150 -> 0.138 | Helped |
| Add convolutional attention (k=6) (ablation) | Full-attention Transformer · electricity-f (1d rolling, same input length) | R0.5: 0.082 -> 0.074 | Helped |
| Add convolutional attention (k=6) (ablation) | LogSparse Transformer · traffic-f (1d rolling, same input length) | R0.5: 0.150 -> 0.138 | Helped |
| Use LogSparse attention plus convolutional attention rather than canonical full attention (ablation) | Transformer · traffic-f (1d rolling, same input length) | R0.5: 0.147 -> 0.138 | Helped |
| Use Transformer rather than DeepAR | Time-series forecasting · Synthetic data as history length increases, especially t0 >= 96 | Rho-quantile loss | Mixed |
Closest papers by meaning
- SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention (ICML 2024 · similarity 0.90)
- Are Self-Attentions Effective for Time Series Forecasting? (NeurIPS 2024 · similarity 0.89)
- A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle (ICML 2025 · similarity 0.88)
- Sequence Complementor: Complementing Transformers for Time Series Forecasting with Learnable Sequences (AAAI 2025 · similarity 0.88)
- iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (ICLR 2024 · similarity 0.88)
- HDT: Hierarchical Discrete Transformer for Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.