Research map / Time series / Foundation models for time series
How Different from the Past? Spatio-Temporal Time Series Forecasting with Self-Supervised Deviation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ST-SSDL, a GCRU-based spatio-temporal forecaster that anchors inputs to historical averages and uses learnable prototypes with contrastive and deviation losses to quantify and exploit current-vs-history deviations.
Problem
Spatio-temporal traffic forecasting often ignores dynamic deviations between current observations and historical patterns, and such deviations are hard to quantify in continuous latent space.
Approach
Anchor each input to its timestamp-aligned historical average, discretize the latent space with learnable prototypes, and enforce relative distance consistency between physical-space deviation and prototype-space distance via a self-supervised deviation loss, alongside a contrastive prototype-separation loss.
Key results
- On METR-LA at 60-min horizon, ST-SSDL reports MAE 3.37, RMSE 7.17, MAPE 9.48%, the best among compared baselines in the table.
- On PEMS07 (12-step average), ST-SSDL reports MAE 19.19 and RMSE 32.51 versus the next best MAE 19.61 (STID) in the table.
- Ablations show removing both L_Con and L_Dev raises PEMS08 RMSE from 23.10 to 24.13 and removing SSDL entirely raises it to 24.96.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the full ST-SSDL deviation-learning framework | GCRU forecasting model · METRLA, PEMSBAY, PEMSD7(M), PEMS04, PEMS07, PEMS08 | RMSE: 6.16 -> 6.02; 3.65 -> 3.51; 5.18 -> 5.11; 29.92 -> 29.64; 33.16 -> 32.51; 23.31 -> 23.10 | Helped |
| add contrastive loss (ablation) | ST-SSDL · METRLA, PEMSBAY, PEMSD7(M), PEMS04, PEMS07, PEMS08 | RMSE: 6.16 -> 6.02; 3.65 -> 3.51; 5.18 -> 5.11; 29.81 -> 29.64; 33.16 -> 32.51; 23.31 -> 23.10 | Helped |
| add deviation loss (ablation) | ST-SSDL · METRLA, PEMSBAY, PEMSD7(M), PEMS04, PEMS07, PEMS08 | RMSE: 6.07 -> 6.02; 3.62 -> 3.51; 5.14 -> 5.11; 29.84 -> 29.64; 32.68 -> 32.51; 24.09 -> 23.10 | Helped |
| add both contrastive and deviation losses (ablation) | ST-SSDL · METRLA, PEMSBAY, PEMSD7(M), PEMS04, PEMS07, PEMS08 | RMSE: 6.06 -> 6.02; 3.67 -> 3.51; 5.16 -> 5.11; 29.91 -> 29.64; 32.81 -> 32.51; 24.13 -> 23.10 | Helped |
| add SSDL with its historical input (ablation) | GCRU forecasting model · METRLA, PEMSBAY, PEMSD7(M), PEMS04, PEMS07, PEMS08 | RMSE: 6.09 -> 6.02; 3.54 -> 3.51; 5.22 -> 5.11; 30.01 -> 29.64; 32.90 -> 32.51; 24.96 -> 23.10 | Helped |
| use prototype discretization instead of naive continuous-space cosine deviation (ablation) | ST-SSDL · METRLA, PEMSBAY | RMSE: 6.07 -> 6.02; 3.52 -> 3.51 | Helped |
Surprising findings
- ST-SSDL has the largest runtime on PEMS07 training (870s per epoch) and inference (217.8s) among compared models, despite having few parameters.
- Removing the stop-gradient operator causes all queries to collapse onto a single prototype (reported in appendix).
Limitations
- Runtime is higher than several baselines due to iterative GCRU backbone, especially on PEMS07.
- Evaluated only on traffic datasets with single-channel data (C=1).
- Hyperparameters (M, d, h) tuned per benchmark; sensitivity shown only on METR-LA and PEMS-BAY.
Experimental setup
Datasets: METR-LA PEMS-BAY PEMSD7(M) PEMS04 PEMS07 PEMS08
Benchmarks: METR-LA PEMS-BAY PEMSD7(M) PEMS04 PEMS07 PEMS08
Metrics: MAE RMSE MAPE
Baselines: HI GRU STGCN DCRNN Graph WaveNet MTGNN AGCRN GTS STNorm STID ST-WA MegaCRN
Builds on: GCRU AGCRN triplet loss
Closest papers by meaning
- ST-ReP: Learning Predictive Representations Efficiently for Spatial-Temporal Forecasting (AAAI 2025 · similarity 0.88)
- Learning with Calibration: Exploring Test-Time Computing of Spatio-Temporal Forecasting (NeurIPS 2025 · similarity 0.88)
- ST-FiT: Inductive Spatial-Temporal Forecasting with Limited Training Data (AAAI 2025 · similarity 0.88)
- Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting (ICLR 2025 · similarity 0.87)
- Causal Spatio-Temporal Prediction: An Effective and Efficient Multi-Modal Approach (NeurIPS 2025 · similarity 0.87)
- Graph-based Forecasting with Missing Data through Spatiotemporal Downsampling (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.