Research map / Time series / Long-term forecasting
Unlocking the Power of Patch: Patch-Based MLP for Long-Term Time Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PatchMLP, an all-MLP long-term time series forecaster that combines multi-scale patch embedding, moving-average decomposition of latent embeddings, and intra/inter-variable MLP mixing with dot-product interaction.
Problem
Long-term time series forecasting (LTSF) with Transformers: whether attention is needed at all, and how to properly model cross-variable interactions in multivariate forecasting.
Approach
Attributes Transformer gains to the patch mechanism rather than attention, and builds a purely MLP model that embeds patches at multiple scales, decomposes the latent embedding (not the raw series) via average pooling into smooth and residual parts, and mixes variables via a dot-product between MLP outputs and inputs.
Key results
- PatchMLP achieves the best MSE and MAE on all 8 datasets (16 metrics), e.g. ETTm1 MSE 0.374 vs 0.381 for TimeMixer and ECL MSE 0.171 vs 0.178 for iTransformer.
- Ablation on ECL/Traffic shows removing multi-scale patch embedding raises ECL MSE from 0.171 to 0.183 and removing the dot-product interaction raises it to 0.177.
- Replacing latent-vector decomposition with conventional raw-series decomposition worsens ECL MSE from 0.171 to 0.180 (case 9 vs case 1).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PatchMLP instead of the strongest baseline | LTSF forecasting · ETTm1 | MSE: 0.381 -> 0.374 | Helped |
| Use PatchMLP instead of the strongest baseline | LTSF forecasting · ETTm2 | MSE: 0.275 -> 0.269 | Helped |
| Use PatchMLP instead of the strongest baseline | LTSF forecasting · ETTh1 | MSE: 0.447 -> 0.438 | Helped |
| Use PatchMLP instead of the strongest baseline | LTSF forecasting · ETTh2 | MSE: 0.364 -> 0.349 | Helped |
| Use PatchMLP instead of the strongest baseline | LTSF forecasting · ECL | MSE: 0.178 -> 0.171 | Helped |
| Use PatchMLP instead of the strongest baseline | LTSF forecasting · Traffic | MSE: 0.428 -> 0.417 | Helped |
| Use PatchMLP instead of the strongest baseline | LTSF forecasting · Weather | MSE: 0.240 -> 0.231 | Helped |
| Use PatchMLP instead of the strongest baseline | LTSF forecasting · Solar-Energy | MSE: 0.216 -> 0.211 | Helped |
| Add MPE (ablation) | PatchMLP · ECL | MSE: 0.183 -> 0.171 | Helped |
| Add the inter-variable dot product instead of a residual connection (ablation) | PatchMLP · ECL | MSE: 0.177 -> 0.171 | Helped |
| Add the inter-variable MLP (ablation) | PatchMLP · ECL | MSE: 0.179 -> 0.171 | Helped |
| Add latent-vector decomposition (ablation) | PatchMLP · ECL | MSE: 0.186 -> 0.171 | Helped |
Surprising findings
- Simple linear layers augmented with patching can outperform complex Transformer-based LTSF models.
- Channel-independent PatchTST underperforms, and Crossformer's explicit cross-variable modeling also underperforms, suggesting cross-variable interaction must be applied carefully.
Limitations
- Main results use a fixed input length of 96 following iTransformer's setting; longer look-back behavior is only analyzed separately.
- Hardware, training cost and run-to-run variance are not reported in the text.
- Performance gains on some datasets are small (e.g. ECL MSE 0.171 vs 0.178), so significance is unclear.
Experimental setup
Datasets: Solar Energy Weather Traffic Electricity (ECL) ETTh1 ETTh2 ETTm1 ETTm2
Benchmarks: Solar Energy Weather Traffic Electricity (ECL) ETTh1 ETTh2 ETTm1 ETTm2
Metrics: MSE MAE
Baselines: TimeMixer iTransformer RLinear PatchTST Crossformer TiDE TimesNet DLinear SCINet FEDformer
Builds on: PatchTST iTransformer DLinear RLinear
Closest papers by meaning
- ARM: Refining Multivariate Forecasting with Adaptive Temporal-Contextual Learning (ICLR 2024 · similarity 0.90)
- HDMixer: Hierarchical Dependency with Extendable Patch for Multivariate Time Series Forecasting (AAAI 2024 · similarity 0.89)
- WPMixer: Efficient Multi-Resolution Mixing for Long-Term Time Series Forecasting (AAAI 2025 · similarity 0.89)
- DeformableTST: Transformer for Time Series Forecasting without Over-reliance on Patching (NeurIPS 2024 · similarity 0.89)
- A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle (ICML 2025 · similarity 0.89)
- xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.