Research map / Time series / Long-term forecasting
ARM: Refining Multivariate Forecasting with Adaptive Temporal-Contextual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ARM, a multivariate long-term forecasting method combining adaptive univariate effect learning (AUEL), random dropping of series, and multi-kernel local smoothing (MKLS) that lets a vanilla Transformer beat prior multivariate and univariate LTSF baselines on 10 datasets.
Problem
Multivariate long-term time series forecasting (LTSF) Transformers underperform univariate models because they mix series with differing characteristics and model inter-series dependencies poorly.
Approach
Disentangle per-series univariate effects first (adaptive EMA output mean/std and MoE-based temporal pattern initialization), then train the multivariate predictor with random dropping of series in input and target and use multi-kernel local convolutions with channel attention to build series-specific temporal representations.
Key results
- ARM (Vanilla) reports best MSE/MAE on most of the 10 benchmarks, e.g. Electricity (96) MSE 0.125 vs PatchTST 0.129 and Multi (96) MSE 0.032 vs DLinear 0.067.
- Adding ARM to vanilla Transformer cuts average MSE across Electricity and ETTm1 from 0.720 to 0.249 (34.6% of original).
- Adding ARM to Autoformer reduces average MSE on the Electricity and ETTm1 subsets from 0.439 to 0.253.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply ARM to the vanilla Transformer | Vanilla Transformer · Electricity, ETTm1, ETTm2, ETTh1, ETTh2, Weather, Traffic, Exchange, ILI and Multi (96-step horizon except ILI 24) | MSE: 0.360 -> 0.125 (Electricity); 0.917 -> 0.287 (ETTm1); 0.078 vs 0.032 (Multi) | Helped |
| Add AUEL only (ablation) | Vanilla Transformer · Electricity (96) | MSE: 0.360 -> 0.130 | Helped |
| Add Random Dropping only (ablation) | Vanilla Transformer · Electricity (96) | MSE: 0.360 -> 0.341 | Helped |
| Add MKLS only (ablation) | Vanilla Transformer · Electricity (96) | MSE: 0.360 -> 0.378 | Hurt |
| Remove AUEL distribution learning (ablation) | ARM (Vanilla) · Electricity (96) | MSE: 0.125 -> 0.132 | Hurt |
| Remove AUEL temporal-pattern learning (ablation) | ARM (Vanilla) · Electricity (96) | MSE: 0.125 -> 0.142 | Hurt |
| Replace temporal-pattern MoE with independent linears (ablation) | AUEL in ARM (Vanilla) · Electricity (96) | MSE: 0.125 -> 0.133 | Hurt |
| Replace temporal-pattern MoE with a single linear (ablation) | AUEL in ARM (Vanilla) · Electricity (96) | MSE: 0.125 -> 0.128 | Mixed |
| Remove Pre-MKLS (ablation) | ARM (Vanilla) · Electricity (96) | MSE: 0.125 -> 0.127 | Hurt |
| Remove Post-MKLS (ablation) | ARM (Vanilla) · Electricity (96) | MSE: 0.125 -> 0.130 | Hurt |
| Remove MKLS channel-wise attention (ablation) | ARM (Vanilla) · Electricity (96) | MSE: 0.125 -> 0.129 | Hurt |
| Use MKLS kernel sizes s=[49 145 385 673] (ablation) | ARM (Vanilla) · Electricity (96) | MSE: 0.125 -> 0.127 | No clear effect |
Surprising findings
- Previous multivariate models such as Autoformer and Informer fail to beat univariate models on the synthetic Multi dataset, which contains simple shift-based inter-series dependencies.
Limitations
- Baseline results are taken from prior papers or rerun, with ARM using L_I=720 while baselines tuned over lookback lengths, so comparisons are not fully matched.
- Dataset-specific choices such as hidden dimension d and kernel sizes must be set by dataset scale, and the paper notes sensitivity to these settings.
Experimental setup
Datasets: ETT Electricity Traffic Weather ILI Exchange Multi
Benchmarks: ETTm1 ETTm2 ETTh1 ETTh2 Electricity Traffic Weather ILI Exchange Multi
Metrics: MSE MAE
Baselines: PatchTST DLinear FEDformer Autoformer Informer Repeat
Builds on: Transformer RevIN NLinear PatchTST DLinear
Closest papers by meaning
- CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables (ICML 2024 · similarity 0.90)
- Unlocking the Power of Patch: Patch-Based MLP for Long-Term Time Series Forecasting (AAAI 2025 · similarity 0.90)
- TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data (AAAI 2025 · similarity 0.89)
- AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting (ICML 2025 · similarity 0.89)
- Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift (NeurIPS 2025 · similarity 0.88)
- Rethinking Post-Training Recipes for Multimodal Time-Series Forecasting (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.