Research map / Time series / Long-term forecasting
Temporal Query Network for Efficient Multivariate Time Series Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Temporal Query (TQ), periodically shifted learnable vectors used as attention queries, and TQNet, a single-layer MHA plus shallow MLP forecaster that reaches state-of-the-art accuracy on 12 MTSF datasets with linear-level efficiency.
Problem
Multivariate time series forecasting needs accurate inter-variable correlation modeling, but per-sample correlations are distorted by non-stationary disturbances such as noise, missing data, and extreme values.
Approach
Use periodically shifted learnable vectors (indexed by t mod W) as attention queries so that global inter-variable correlations are captured, while keys and values come from the raw input to keep sample-level information; combined with a single-layer MHA and a shallow MLP.
Key results
- TQNet ranks in the top 2 on 22 of 24 MSE/MAE metrics averaged over horizons across 12 datasets (look-back 96).
- On PEMS03 TQNet reaches average MSE 0.097 and MAE 0.203, versus 0.112/0.214 for the next best TimeXer.
- Adding the TQ technique to iTransformer lowers Electricity average MSE from 0.175 to 0.163 and PEMS04 from 0.120 to 0.091 in the integration study.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TQNet instead of TimeXer | ETTh1 forecasting · ETTh1 (4-horizon average) | MSE: 0.437 -> 0.441 | Hurt |
| Use TQNet instead of TimeXer | ETTh2 forecasting · ETTh2 (4-horizon average) | MSE: 0.368 -> 0.378 | Hurt |
| Use TQNet instead of CycleNet | ETTm1 forecasting · ETTm1 (4-horizon average) | MSE: 0.379 -> 0.377 | Helped |
| Use TQNet instead of CycleNet | ETTm2 forecasting · ETTm2 (4-horizon average) | MSE: 0.266 -> 0.277 | Hurt |
| Use TQNet instead of CycleNet | Electricity forecasting · Electricity (4-horizon average) | MSE: 0.168 -> 0.164 | Helped |
| Use TQNet instead of CycleNet | Solar forecasting · Solar (4-horizon average) | MSE: 0.210 -> 0.198 | Helped |
| Use TQNet instead of iTransformer | Traffic forecasting · Traffic (4-horizon average) | MSE: 0.428 -> 0.445 | Hurt |
| Use TQNet instead of TimeXer | Weather forecasting · Weather (4-horizon average) | MSE: 0.241 -> 0.242 | Hurt |
| Use TQNet instead of TimeXer | PEMS03 forecasting · PEMS03 (4-horizon average) | MSE: 0.112 -> 0.097 | Helped |
| Use TQNet instead of TimeXer | PEMS04 forecasting · PEMS04 (4-horizon average) | MSE: 0.105 -> 0.091 | Helped |
| Use TQNet instead of TimeXer | PEMS07 forecasting · PEMS07 (4-horizon average) | MSE: 0.085 -> 0.075 | Helped |
| Use TQNet instead of iTransformer | PEMS08 forecasting · PEMS08 (4-horizon average) | MSE: 0.150 -> 0.142 | Helped |
Surprising findings
- Removing the TQ module (keeping MHA and MLP) degrades performance more than removing the attention mechanism, indicating the TQ queries are the key component.
- Setting W to 1 (no real periodicity) still gives gains over no TQ, which the authors attribute to acting as a weak channel identifier.
Limitations
- Relies on inherent data periodicity to choose W, which may limit generalization to non-periodic data.
- Enforcing multivariate modeling can add complexity and hurt performance when inter-variable correlations are weak.
- Benefit of multivariate modeling diminishes with long look-back windows.
Experimental setup
Datasets: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Solar Traffic Weather PEMS03 PEMS04 PEMS07 PEMS08
Benchmarks: ETTh1 ETTh2 ETTm1 ETTm2 Electricity Solar Traffic Weather PEMS03 PEMS04 PEMS07 PEMS08
Metrics: MSE MAE
Baselines: TimeXer CycleNet iTransformer MSGNet TimesNet PatchTST Crossformer DLinear SCINet
Builds on: iTransformer CycleNet PatchTST DLinear RevIN
Closest papers by meaning
- TimePFN: Effective Multivariate Time Series Forecasting with Synthetic Data (AAAI 2025 · similarity 0.88)
- ARM: Refining Multivariate Forecasting with Adaptive Temporal-Contextual Learning (ICLR 2024 · similarity 0.87)
- CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables (ICML 2024 · similarity 0.86)
- TStruct: Learning Shared Temporal Structures for Long-Term Time-Series Forecasting (NeurIPS 2026 · similarity 0.86)
- Revisiting Attention for Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.86)
- TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.