Research map / Time series / Long-term forecasting
TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TimeXer, an unmodified canonical Transformer that embeds endogenous series as patch tokens plus a global token and exogenous series as variate tokens, combining patch-wise self-attention with variate-wise cross-attention for forecasting with exogenous variables.
Problem
Time series forecasting when external (exogenous) variables are available; existing deep models either treat all variables equally (multivariate) or ignore exogenous information, and struggle with misaligned, missing, or heterogeneous exogenous series.
Approach
Embed endogenous series at patch granularity with a learnable global token, and each exogenous series as a single variate token; apply self-attention over endogenous patches plus global token, and cross-attention from the global token to exogenous variate tokens, bridging granularities without modifying the Transformer.
Key results
- On short-term EPF forecasting, TimeXer achieves average MSE 0.307 and MAE 0.265, outperforming the best baseline iTransformer (0.335/0.289) and others.
- On long-term multivariate benchmarks, TimeXer achieves best MSE/MAE on ECL (0.171/0.270), Weather (0.241/0.271), ETTh1, ETTh2, ETTm1, and ETTm2, but is behind iTransformer on Traffic MSE (0.466 vs 0.428).
- Ablation shows replacing variate-level exogenous tokens with patch tokens or removing the global token degrades average EPF MSE from 0.307 to 0.312 and 0.316 respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TimeXer instead of iTransformer | EPF · EPF AVG | MSE: 0.335 -> 0.307 | Helped |
| use TimeXer instead of iTransformer | multivariate forecasting · Traffic | MSE: 0.428 -> 0.466 | Hurt |
| add an endogenous global token (ablation) | TimeXer · EPF AVG | MSE: 0.316 -> 0.307 | Helped |
| use variate-level rather than patch-level exogenous embeddings (ablation) | TimeXer · EPF AVG | MSE: 0.312 -> 0.307 | Helped |
| use cross-attention instead of adding exogenous tokens to endogenous tokens (ablation) | TimeXer · EPF AVG | MSE: 0.329 -> 0.307 | Helped |
| use cross-attention instead of concatenating tokens (ablation) | TimeXer · EPF AVG | MSE: 0.312 -> 0.307 | Helped |
| use non-overlapping rather than overlapping endogenous patches (ablation) | TimeXer · EPF AVG | MSE: 0.316 -> 0.307 | Helped |
| use variate-level rather than patch-level exogenous embeddings (ablation) | TimeXer · ETTh1 Avg | MSE: 0.075 -> 0.073 | Helped |
| add an endogenous global token (ablation) | TimeXer · ETTh1 Avg | MSE: 0.076 -> 0.073 | Helped |
| add exogenous variate tokens to endogenous variate tokens (ablation) | TimeXer · ETTh1 Avg | MSE: 0.073 -> 0.078 | Hurt |
| concatenate exogenous variate and endogenous temporal tokens for self-attention (ablation) | TimeXer · Traffic Avg | MSE: 0.156 -> 0.149 | Helped |
| replace informative exogenous series with zero-filled series (ablation) | TimeXer · EPF AVG | MSE: 0.307 -> 0.330 | Hurt |
Surprising findings
- Zero-filled or random-noise exogenous series cause little performance drop, whereas corrupting the endogenous series causes a large decline, suggesting the endogenous branch dominates predictions.
- On Traffic, TimeXer's MSE trails iTransformer and PatchTST considerably despite comparable MAE, attributed to failing to predict future spike magnitudes.
Limitations
- Self-attention over patches remains quadratic in look-back/patch count; complexity not reduced without replacing full attention.
- Underperforms iTransformer and PatchTST on Traffic MSE, attributed to patch-level tokens overwhelming the single global token for predicting spikes.
Experimental setup
Datasets: EPF (NP, PJM, BE, FR, DE) ECL Weather ETTh1 ETTh2 ETTm1 ETTm2 Traffic ERA5 weather data (large-scale scalability study) NCEI hourly temperature (large-scale scalability study)
Benchmarks: EPF (NP, PJM, BE, FR, DE) ECL Weather ETTh1 ETTh2 ETTm1 ETTm2 Traffic
Metrics: MSE MAE
Baselines: iTransformer PatchTST Crossformer Autoformer TimesNet SCINet RLinear DLinear TiDE
Builds on: PatchTST iTransformer Vision Transformer
Closest papers by meaning
- A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle (ICML 2025 · similarity 0.90)
- iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (ICLR 2024 · similarity 0.89)
- Timer-XL: Long-Context Transformers for Unified Time Series Forecasting (ICLR 2025 · similarity 0.89)
- TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop (NeurIPS 2025 · similarity 0.88)
- OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain (NeurIPS 2025 · similarity 0.88)
- TimePerceiver: An Encoder-Decoder Framework for Generalized Time-Series Forecasting (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.