Research map / Time series / Long-term forecasting
Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pathformer is a multi-scale Transformer that combines multiple patch sizes (temporal resolution) with dual intra/inter-patch attention (temporal distance), and uses a seasonality/trend-aware router with top-K selection to adaptively choose patch scales per input.
Problem
Transformers for time series forecasting model series at fixed or limited scales, and cannot adaptively choose which temporal scales matter for different series.
Approach
Unifies multi-scale temporal resolution (multiple patch sizes) with temporal distance (intra-patch local and inter-patch global dual attention), and adds a noisy top-K multi-scale router informed by DFT seasonality and trend decomposition that adaptively activates and weights patch-size pathways per layer.
Key results
- Pathformer achieves best MSE/MAE in 81 of 88 multivariate settings (input length 96), with 8.1% MSE and 6.4% MAE reduction versus the second-best PatchTST.
- In transfer learning (ETTh1/ETTm1 to ETTh2/ETTm2 and Cluster pretraining), part-tuning cuts training time by about 52% while staying close to full-tuning accuracy.
- Ablation shows removing the pathways (using all patch sizes) degrades accuracy, and K=2 or K=3 selected patch sizes performs best.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use adaptive multi-scale Pathformer instead of PatchTST | time series forecasting · 11 real-world datasets, multivariate forecasting | MSE: 8.1% reduction in MSE | Helped |
| remove inter-patch attention (ablation) | Pathformer · Weather, prediction 96 | MSE: 0.156 -> 0.162 | Hurt |
| remove intra-patch attention (ablation) | Pathformer · Weather, prediction 96 | MSE: 0.156 -> 0.170 | Hurt |
| remove time series decomposition (ablation) | Pathformer · Weather, prediction 96 | MSE: 0.156 -> 0.162 | Hurt |
| use all patch sizes instead of adaptive pathways (ablation) | Pathformer · Weather, prediction 96 | MSE: 0.156 -> 0.168 | Hurt |
| select K=2 rather than K=1 patch size (ablation) | Pathformer · ETTh2, prediction 96 | MSE: 0.283 -> 0.279 | Helped |
| select K=3 rather than K=2 patch sizes (ablation) | Pathformer · Electricity, prediction 336 | MSE: 0.186 -> 0.181 | Helped |
Surprising findings
- Removing adaptive pathways and using all patch sizes markedly hurts accuracy, suggesting more scales can add redundant signals.
Limitations
- Evaluation limited to standard benchmarks plus a private cloud dataset; the cloud cluster data is not public.
- Scaleformer and other multi-scale baselines are compared under a shared setup, with fixed input length 96 for main results.
- Pathways add router and multi-patch-size computation, increasing model complexity.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity Traffic ILI Cluster-A Cluster-B Cluster-C PEMS07
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity Traffic ILI Cluster-A Cluster-B Cluster-C
Metrics: MSE MAE
Baselines: PatchTST NLinear Scaleformer TiDE FEDformer Pyraformer Autoformer DLinear N-HiTS
Builds on: PatchTST Mixture of Experts Instance Normalization
Closest papers by meaning
- SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention (ICML 2024 · similarity 0.88)
- DeformableTST: Transformer for Time Series Forecasting without Over-reliance on Patching (NeurIPS 2024 · similarity 0.87)
- Revisiting the Generic Transformer: Deconstructing a Strong Baseline for Time Series Foundation Models (NeurIPS 2026 · similarity 0.86)
- Ada-MSHyper: Adaptive Multi-Scale Hypergraph Transformer for Time Series Forecasting (NeurIPS 2024 · similarity 0.86)
- A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle (ICML 2025 · similarity 0.85)
- Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patching (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.