Research map / Graph learning / Knowledge and temporal graphs
Efficient Traffic Prediction Through Spatio-Temporal Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Distills a high-capacity spatio-temporal GNN teacher into a lightweight MLP student (LightST) using prediction-level KL/MSE distillation and contrastive embedding alignment, giving 5-40x faster traffic forecasting with better accuracy.
Problem
Traffic flow prediction with spatio-temporal GNNs is computationally expensive at inference due to high-order message passing, and deep GNNs suffer from over-smoothing that makes region representations indistinguishable.
Approach
Dual-level spatio-temporal knowledge distillation from a GNN+TCN teacher to a graph-free MLP student, combining prediction-level soft-label distillation with adaptive contrastive embedding alignment (spatial and temporal) to avoid uniform alignment and mitigate over-smoothing.
Key results
- On PeMSD7, LightST reaches MAE 20.78 and RMSE 33.95, versus 21.42/34.51 for DSTAGNN, with inference 2.51s vs 110.06s (about 44x faster).
- On PeMS-Bay, LightST achieves MAE 1.78 and RMSE 3.88, the best among compared methods, with inference 5.12s vs 208.51s for STGODE (about 40x faster).
- Abstract claims 5x to 40x inference speedup over state-of-the-art spatio-temporal GNNs while maintaining superior accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LightST instead of the strongest listed baseline | LightST · PeMS-Bay | MAE: 1.95 -> 1.78 | Helped |
| Use LightST instead of the strongest listed baseline | LightST · PeMS-Bay | RMSE: 4.52 -> 3.88 | Helped |
| Use LightST instead of the strongest listed baseline | LightST · PeMSD7 | MAPE: 8.95% -> 8.98% | Hurt |
| Add embedding-level knowledge distillation (ablation) | LightST · four traffic datasets | prediction results | Helped |
| Add spatial adaptive embedding alignment (L^(P)) (ablation) | LightST · four traffic datasets | prediction results | Helped |
| Add temporal embedding alignment (L^(E)) (ablation) | LightST · four traffic datasets | prediction results | Helped |
Surprising findings
- A graph-free MLP student outperforms the GNN teachers and many GNN baselines on several datasets, despite not doing graph message passing at inference.
Limitations
- Relies on a GNN teacher that must be trained first, so training cost is not eliminated.
- Adaptive weighting justification is a heuristic gradient analysis rather than a formal guarantee against over-smoothing.
- Text describes the paper as consistently best 'in most cases', and some reported results (e.g., PeMSD4 MAE) are not best.
Experimental setup
Datasets: PeMSD3 PeMSD4 PeMSD7 PeMSD8 PeMS-Bay
Benchmarks: PeMS-Bay PeMSD4 PeMSD8 PeMSD3 PeMSD7
Metrics: MAE RMSE MAPE inference time
Baselines: HA VAR DSANet DCRNN STGCN GWN ASTGCN LSTGCN STSGCN AGCRN STFGNN STGODE
Builds on: STKD KD-Pruning STID AGCRN GWN
Closest papers by meaning
- Spatio-Temporal Pivotal Graph Neural Networks for Traffic Flow Forecasting (AAAI 2024 · similarity 0.91)
- DSTAGNN: Dynamic Spatial-Temporal Aware Graph Neural Network for Traffic Flow Forecasting (ICML 2022 · similarity 0.91)
- TESTAM: A Time-Enhanced Spatio-Temporal Attention Model with Mixture of Experts (ICLR 2024 · similarity 0.89)
- Spatiotemporal-aware Trend-Seasonality Decomposition Network for Traffic Flow Forecasting (AAAI 2025 · similarity 0.89)
- Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective (AAAI 2024 · similarity 0.89)
- Decomposed Spatio-Temporal Mamba for Long-Term Traffic Prediction (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.