Research map / Graph learning / Knowledge and temporal graphs
TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TESTAM, a mixture-of-experts traffic forecaster with identity, static-graph, and attention spatial experts, routed by a memory-based gate trained with pseudo-label classification losses.
Problem
Traffic forecasting must capture both recurring and non-recurring (event-driven) spatio-temporal patterns, but existing models use one fixed spatial modeling method and do not adapt the method to traffic context.
Approach
Mixture of three experts with different spatial modeling (no spatial modeling, learnable static graph, dynamic attention) and a memory-based gating network trained with two classification losses (worst-route avoidance and best-route selection) using regression-error pseudo labels; also adds Time2Vec temporal embedding and time-enhanced attention that removes autoregressive decoding.
Key results
- On METR-LA at 60 min, TESTAM reports MAE 3.36, RMSE 7.09, MAPE 9.67%, the best among 13 baselines in that table.
- On PEMS-BAY 60 min, TESTAM reports MAE 1.85, RMSE 4.33, MAPE 4.31%, the best among the compared models.
- On EXPY-TKY 60 min, TESTAM reports MAE 6.75, RMSE 11.01, MAPE 31.01%, with the best values at 30 and 60 min.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add in-situ gating over diverse experts (ablation) | TESTAM · METR-LA average prediction windows | MAE: 3.00 -> 2.93 | Helped |
| use top-1 expert routing instead of weighted-sum ensemble (ablation) | TESTAM · METR-LA average prediction windows | MAE: 2.98 -> 2.93 | Helped |
| add best-route selection loss (ablation) | TESTAM · METR-LA average prediction windows | MAE: 2.96 -> 2.93 | Helped |
| preserve the identity expert rather than replace it with a GCN-based adaptive expert (ablation) | TESTAM · METR-LA average prediction windows | MAE: 2.97 -> 2.93 | Helped |
| use temporal information embedding with periodic activation functions instead of simple embedding vectors (ablation) | TESTAM · METR-LA average prediction windows | MAE: 2.96 -> 2.93 | Helped |
| use time-enhanced attention instead of basic temporal attention (ablation) | TESTAM · METR-LA average prediction windows | MAE: 2.99 -> 2.93 | Helped |
| use TESTAM instead of MegaCRN | traffic forecasting · METR-LA 60 min | MAE: 3.38 -> 3.36 | Helped |
| use TESTAM instead of MegaCRN | traffic forecasting · PEMS-BAY 60 min | MAE: 1.88 -> 1.85 | Helped |
| use TESTAM instead of MegaCRN | traffic forecasting · EXPY-TKY 60 min | MAE: 6.83 -> 6.75 | Helped |
Surprising findings
- Removing the gating mechanism (w/o gating, attention expert only) yields performance close to GMAN, suggesting the gain comes from in-situ modeling rather than the base architecture.
- Replacing the identity expert with a GCN-based adaptive expert performs worse than even worst-route avoidance-only training, indicating diverse graph structures matter.
Limitations
- Baseline results partly taken from original papers rather than re-run, and StemGNN has no PEMS-BAY results for 30/60 min.
- Case-study road subsets (isolated, hard-to-predict, event) are selected by heuristics and visual inspection, which may not be reproducible.
- Attention-based experts can spread weight uniformly; time-varying graphs are noise sensitive.
Experimental setup
Datasets: METR-LA PEMS-BAY EXPY-TKY
Benchmarks: METR-LA PEMS-BAY EXPY-TKY
Metrics: MAE RMSE MAPE
Baselines: HA STGCN DCRNN Graph-WaveNet STTN GMAN MTGNN StemGNN AGCRN CCRNN GTS PM-MemNet
Builds on: SMoEs MegaCRN GMAN Graph-WaveNet Time2Vec
Closest papers by meaning
- Dynamic Spatial-Temporal Graph Convolutional Neural Networks for Traffic Forecasting (AAAI 2019 · similarity 0.90)
- DSTAGNN: Dynamic Spatial-Temporal Aware Graph Neural Network for Traffic Flow Forecasting (ICML 2022 · similarity 0.90)
- Efficient Traffic Prediction Through Spatio-Temporal Distillation (AAAI 2025 · similarity 0.89)
- Spatio-Temporal Pivotal Graph Neural Networks for Traffic Flow Forecasting (AAAI 2024 · similarity 0.89)
- Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective (AAAI 2024 · similarity 0.87)
- Spatiotemporal-aware Trend-Seasonality Decomposition Network for Traffic Flow Forecasting (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.