Research map / Video / Video segmentation and tracking
SSL-STMFormer Self-Supervised Learning Spatio-Temporal Entanglement Transformer for Traffic Flow Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SSL-STMFormer, a transformer for traffic flow prediction that combines spatio-temporal attention with masking-based entanglement-aware modules and self-supervised heterogeneity losses, reporting gains on six PeMS, NYC Taxi, CHI Bike and T-Drive datasets.
Problem
Traffic flow prediction struggles to capture long-range, long-time similarities, spatio-temporal heterogeneity, and the dynamic entanglement between space and time in traffic data.
Approach
A spatio-temporal transformer whose attention is masked by random, tube, block, short-range (hop-based) and long-range (DTW-based) masks, combined with adaptive augmentation and self-supervised spatial (soft clustering) and temporal (contrastive) heterogeneity losses.
Key results
- On graph-based datasets, the model reports average improvements of 5.04% MAE, 5.53% MAPE and 5.02% RMSE over the next best results (e.g., PeMS08 MAE 12.05 vs 13.58 for PDFormer).
- On grid-based datasets, average improvements of 10.91% MAE, 9.86% MAPE and 7.18% RMSE over the next best results (e.g., T-Drive inflow MAE 12.85 vs 14.93 for STD-MAE).
- Ablation shows removing SSL raises PeMS08 MAE from 12.05 to 13.70, and replacing separate spatial/temporal entanglement-aware modules with a hybrid one raises it to 13.86.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SSL-STMFormer instead of the strongest baseline | traffic flow prediction · PeMS04 | MAE: 17.80 -> 17.06 | Helped |
| use SSL-STMFormer instead of the strongest baseline | traffic flow prediction · PeMS07 | MAE: 18.65 -> 18.53 | Helped |
| use SSL-STMFormer instead of the strongest baseline | traffic flow prediction · PeMS08 | MAE: 13.44 -> 12.05 | Helped |
| use SSL-STMFormer instead of the strongest baseline | traffic flow prediction · NYCTaxi inflow | MAE: 10.71 -> 9.49 | Helped |
| use SSL-STMFormer instead of the strongest baseline | traffic flow prediction · NYCTaxi outflow | MAE: 9.53 -> 8.98 | Helped |
| use SSL-STMFormer instead of the strongest baseline | traffic flow prediction · T-Drive inflow | MAE: 15.42 -> 12.85 | Helped |
| use SSL-STMFormer instead of the strongest baseline | traffic flow prediction · T-Drive outflow | MAE: 14.93 -> 12.79 | Helped |
| use SSL-STMFormer instead of the strongest baseline | traffic flow prediction · CHIBike inflow | MAE: 3.91 -> 3.49 | Helped |
| use SSL-STMFormer instead of the strongest baseline | traffic flow prediction · CHIBike outflow | MAE: 3.83 -> 3.44 | Helped |
| add self-supervised learning (ablation) | SSL-STMFormer · PeMS08 | MAE: 13.70 -> 12.05 | Helped |
| add spatial and temporal entanglement-aware modules (ablation) | SSL-STMFormer · PeMS08 | MAE: 13.59 -> 12.05 | Helped |
| add the spatial entanglement-aware module (ablation) | SSL-STMFormer · PeMS08 | MAE: 13.46 -> 12.05 | Helped |
Limitations
- Only one hour input and output horizon evaluated on graph datasets; grid datasets use a single-step horizon.
- Improvements on some grid datasets (e.g., PeMS07 at 0.64% MAE) are small.
- Hyperparameter sensitivity studied only on PeMS08 and T-Drive.
Experimental setup
Datasets: PeMS04 PeMS07 PeMS08 NYCTaxi CHIBike T-Drive
Benchmarks: PeMS04 PeMS07 PeMS08 NYCTaxi CHIBike T-Drive
Metrics: MAE MAPE RMSE
Baselines: DCRNN STGCN GWNET MTGNN STSGCN STFGNN STGODE STGNCDE STTN GMAN TFormer ASTGNN
Builds on: ST-SSL STD-MAE Masked Autoencoders Graph Laplacian eigenvectors embedding Dynamic Time Warping
Closest papers by meaning
- Decomposed Spatio-Temporal Mamba for Long-Term Traffic Prediction (AAAI 2025 · similarity 0.89)
- Efficient Traffic Prediction Through Spatio-Temporal Distillation (AAAI 2025 · similarity 0.88)
- Spatiotemporal-aware Trend-Seasonality Decomposition Network for Traffic Flow Forecasting (AAAI 2025 · similarity 0.86)
- Spatio-Temporal Pivotal Graph Neural Networks for Traffic Flow Forecasting (AAAI 2024 · similarity 0.86)
- SEPT: Towards Efficient Scene Representation Learning for Motion Prediction (ICLR 2024 · similarity 0.85)
- Bridging Traffic State and Trajectory for Dynamic Road Network and Trajectory Representation Learning (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.