Research map / Vision-language and multimodal / Multimodal LLMs

DriveSpatial: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces DriveSpatial, a 15.6K human-verified QA benchmark over 20 tasks from five AD datasets, generated from a dynamic multi-relational scene graph, and evaluates 15 VLMs to expose a large human-model gap in multi-view scene construction.

Problem

Existing autonomous driving VLM benchmarks are single-view, static, ego-centric, or single-source, so it is unclear whether VLMs can construct and reason over multi-view, dynamic driving scenes.

Approach

Generates QA pairs from a dynamic multi-relational scene graph (spatial, interaction, action, temporal edges) with camera-visibility constraints that force cross-view and spatiotemporal reasoning, followed by human verification, and organizes tasks into scene construction, multi-view relational understanding, temporal reasoning, and generalization.

Key results

• The strongest model GPT-5 reaches 54.98% on DriveSpatial-mini versus 83.39% for humans, a 28.41-point gap.
• Qwen3-VL-8B is the best open-source model at 42.24% average accuracy.
• BEV map grounding raises average accuracy by 8.27 points for Qwen3-VL, 4.95 for Gemma-3 and 4.55 for Ego3D.

Surprising findings

• Generalist VLMs outperform driving and 3D-embodied specialist models, reversing the trend on earlier AD benchmarks.
• Chain-of-thought, tree-of-thoughts and self-consistency all degrade performance relative to zero-shot prompting.

Limitations

• Human-verified QA still has only 76.86% pass rate, and inter-model agreement with humans is low (kappa at most 0.43).
• Evaluation covers only static scene graphs derived from validation sets, with limited coverage of lighting and sensor modalities such as LiDAR.

Experimental setup

Datasets: nuScenes Waymo AV2 TruckScenes ONCE

Benchmarks: DriveSpatial DriveSpatial-mini

Metrics: accuracy RMSE Cohen's kappa

Baselines: GPT-5 Qwen3-VL-8B Gemma-3 Ego3D RoboTron-Drive SpaceThinker SenseNova-SI

Builds on: nuScenes-QA DriveLM OmniDrive Ego3D-Bench STSBench

Closest papers by meaning

• STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving (NeurIPS 2025 · similarity 0.92)
• LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.92)
• SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Driving (CVPR 2026 · similarity 0.92)
• SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving (CVPR 2026 · similarity 0.91)
• FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving (NeurIPS 2025 · similarity 0.91)
• DriveCombo: Benchmarking Compositional Traffic Rule Reasoning in Autonomous Driving (CVPR 2026 · similarity 0.90)