Research map / Architectures and efficiency / RNNs and normalization
Information Bottleneck-guided MLPs for Robust Spatial-temporal Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RSTIB, a generalized information bottleneck principle for robust spatial-temporal forecasting, and RSTIB-MLP, an efficient MLP instantiation with a teacher-guided knowledge distillation training regime.
Problem
Spatial-temporal forecasting models are vulnerable to noise in both input and target windows (the "dual noise effect"), and efficient MLP-based models lack robustness, while STGNNs are computationally expensive.
Approach
Generalizes RGIB to spatial-temporal data by lifting the Z-X-Y Markov assumption of the information bottleneck, adding input/target/representation KL regularizers via data reparameterization, and weighting them per time series by a teacher-derived noise impact indicator.
Key results
- On PEMS04 at 50% noise, RSTIB-MLP achieves MAE 27.16 versus 29.24 for STGKD and 27.90 for TrendGCN, while being more efficient than STGNNs.
- On Weather2K-R at 50% noise, RSTIB-MLP reports MAE 7073.69 versus 7108.36 for TrendGCN (best baseline), with consistent gains across noise ratios.
- Ablation shows removing the RSTIB instantiation or the knowledge distillation module degrades performance under noise; feature variance is higher than STID under noise (Fig. 3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RSTIB-MLP instead of the strongest listed baseline | Spatial-temporal forecasting · PEMS04, 50% noise | MAE: 27.90 -> 27.16 | Helped |
| Use RSTIB-MLP instead of the strongest listed baseline | Spatial-temporal forecasting · PEMS07, 50% noise | MAE: 32.16 -> 30.94 | Helped |
| Use RSTIB-MLP instead of the strongest listed baseline | Spatial-temporal forecasting · PEMS08, 50% noise | MAE: 25.93 -> 24.37 | Helped |
| Use RSTIB-MLP instead of the strongest listed baseline | Spatial-temporal forecasting · LargeST(SD), 50% noise | MAE: 35.21 -> 32.78 | Helped |
| Use RSTIB-MLP instead of the strongest listed baseline | Spatial-temporal forecasting · Weather2K-R, 50% noise | MAE: 7103.92 -> 7073.69 | Helped |
| Use RSTIB-MLP instead of the strongest listed baseline | Spatial-temporal forecasting · Electricity, 50% noise | MAE: 47.26 -> 45.30 | Helped |
| Add IB-based enhancement (ablation) | RSTIB-MLP · Spatial-temporal forecasting under noise | MAE | Helped |
| Use vanilla IB instead of RSTIB (ablation) | RSTIB-MLP · Spatial-temporal forecasting under noise | MAE | Hurt |
| Add the knowledge distillation module (ablation) | RSTIB-MLP · Spatial-temporal forecasting under noise | MAE | Helped |
Surprising findings
- Vanilla IB instantiation sometimes performs worse than no IB enhancement, which the authors attribute to ignoring target-side noise and difficulty balancing objective terms.
- RSTIB-MLP is less efficient than STID (the plain MLP baseline) in training time per epoch, though far faster than STGNNs.
Limitations
- RSTIB-MLP trains more slowly than the plain STID MLP baseline.
- Gains over robust baselines are often marginal on clean data and some settings show comparable rather than better results.
- Relies on a pretrained teacher model (default STGCN) to compute noise impact indicators.
Experimental setup
Datasets: PEMS04 PEMS07 PEMS08 LargeST(SD) Weather2K-R Electricity
Benchmarks: PEMS04 PEMS07 PEMS08 LargeST(SD) Weather2K-R Electricity
Metrics: MAE RMSE MAPE feature variance seconds/epoch
Baselines: STID GWN TrendGCN STExplainer STExplainer-CGIB STGKD BiTGraph STC-Dropout STG-NCDE FreTS DSTAGNN STHMLP
Builds on: Robust Graph Information Bottleneck (RGIB) Deep Variational Information Bottleneck STID STGKD spatial-temporal prompts
Closest papers by meaning
- ST-FiT: Inductive Spatial-Temporal Forecasting with Limited Training Data (AAAI 2025 · similarity 0.88)
- Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting (ICLR 2025 · similarity 0.86)
- How Different from the Past? Spatio-Temporal Time Series Forecasting with Self-Supervised Deviation Learning (NeurIPS 2025 · similarity 0.85)
- Towards Dynamic Spatial-Temporal Graph Learning: A Decoupled Perspective (AAAI 2024 · similarity 0.85)
- ST-ReP: Learning Predictive Representations Efficiently for Spatial-Temporal Forecasting (AAAI 2025 · similarity 0.85)
- Learning with Calibration: Exploring Test-Time Computing of Spatio-Temporal Forecasting (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.