Research map / Time series / Long-term forecasting
A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add patching (P=16, S=8) to the Transformer (ablation) | PatchTST/42 (channel-independent) · Weather, Traffic, Electricity (horizon 96) | MSE: Weather 0.164->0.152; Traffic 0.397->0.367; Electricity 0.136->0.130 | Helped |
| Add channel-independence to patched Transformer (ablation) | PatchTST/42 (patched) · Weather, Traffic, Electricity (horizon 96) | MSE: Weather 0.168->0.152; Traffic 0.595->0.367; Electricity 0.196->0.130 | Helped |
| Use patching and channel independence together vs. original TST (ablation) | PatchTST/42 · Weather horizon 96 | MSE: 0.177 -> 0.152 | Helped |
| Increase look-back window from 96 to 336 (no patching) (ablation) | PatchTST input · Traffic horizon 96 | MSE: 0.518 -> 0.397 (Weather/Traffic case study, Table 1) | Helped |
| Add patching to L=336 input (fewer tokens, longer history) (ablation) | PatchTST channel-independent · Traffic horizon 96 case study | MSE: 0.397 -> 0.367 | Helped |
| Self-supervised masked pretraining before supervised fine-tuning (ablation) | PatchTST · Traffic horizon 96 (Table 1 case study) | MSE: 0.367 -> 0.349 | Helped |
| Remove instance normalization (ablation) | PatchTST/42 · Weather horizon 96 | MSE: 0.152 with IN vs 0.156 without | Helped |
| Self-supervised pretraining + end-to-end fine-tuning vs supervised training from scratch on large datasets | PatchTST · Traffic horizon 96 (Table 6) | MSE: 0.352 fine-tuning vs 0.410 DLinear and 0.367 supervised PatchTST | Helped |
| Transfer representation pretrained on Electricity to other datasets | PatchTST · Weather, Traffic transfer | MSE: transfer fine-tuning slightly worse than same-dataset pretraining and sometimes worse than supervised | Mixed |
Closest papers by meaning
- SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention (ICML 2024 · similarity 0.91)
- DeformableTST: Transformer for Time Series Forecasting without Over-reliance on Patching (NeurIPS 2024 · similarity 0.90)
- Unlocking the Power of Patch: Patch-Based MLP for Long-Term Time Series Forecasting (AAAI 2025 · similarity 0.90)
- iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (ICLR 2024 · similarity 0.89)
- A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle (ICML 2025 · similarity 0.89)
- Multi-Patch Prediction: Adapting Language Models for Time Series Representation Learning (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.