Research map / Vision-language and multimodal / Multimodal LLMs
ST-VLM: A Spatial-to-Image Multimodal Spatial-Temporal Prediction Framework with Vision-Language Model
Tong Zhao, Junping Du, Zhe Xue, Meiyu Liang, Aijing Li, Xiaolong Meng, Dandan Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ST-VLM, a framework that converts spatial-temporal prediction inputs into images so a vision-language model can reason over spatial and temporal features.
Problem
Spatial-temporal prediction (e.g., traffic, environmental monitoring) where LLM-based forecasters have limited understanding of spatial features, which existing methods tend to oversimplify.
Approach
Uses a spatial-to-image transformation so spatial-temporal data is represented in image form and processed by a vision-language model, rather than treating spatial structure in simplified text/series form.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| spatial-to-image plus VLM fusion vs best LLM baseline STD-PLM | ST-VLM · PEMS03 | MAE: 14.61 -> 11.66 | Improved |
| spatial-to-image plus VLM fusion vs STD-PLM | ST-VLM · PEMS08 | MAE: 13.33 -> 12.23 | Improved |
| ST-VLM vs STD-PLM at 5% training data | ST-VLM · PEMS04 few-shot | MAE: 27.81 -> 24.43 | Improved |
| ST-VLM vs STD-PLM at 20% training data | ST-VLM · PEMS08 few-shot | MAE: 16.49 -> 14.08 | Improved |
| add VLM fine-tuning (LoRA) (ablation) | ST-VLM · PEMS03/PEMS04 | MAE/RMSE | Improved |
| add self-attention-aware fusion on top of LoRA fine-tuning (ablation) | ST-VLM · PEMS03/PEMS04 | MAE/RMSE | No clear effect |
Closest papers by meaning
- Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting (ICML 2025 · similarity 0.90)
- GTR-Bench: Evaluating Geo-Temporal Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.89)
- Causal Spatio-Temporal Prediction: An Effective and Efficient Multi-Modal Approach (NeurIPS 2025 · similarity 0.87)
- DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understanding (NeurIPS 2025 · similarity 0.87)
- LLaVA-ST: A Multimodal Large Language Model for Fine-Grained Spatial-Temporal Understanding (CVPR 2025 · similarity 0.86)
- SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
