Research map / Vision-language and multimodal / Robot action and navigation

Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving

NeurIPS 2026 · Vision-language and multimodal › Robot action and navigation · approach: Vision-language-action foundation models

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GeoCoTDrive, a VLA driving planner that grounds planning-relevant 2D regions, samples localized 3D features from a frozen geometric foundation model (Depth Anything V3 / VGGT) within those regions, and interleaves them into the autoregressive context for trajectory generation, plus a 146K-annotation PlanningGrounding dataset.

Problem

VLA models for autonomous driving reason in a 2D semantic space while driving actions require precise 3D geometry, and existing geometry integration injects global scene-level features rather than local decision-critical cues.

Approach

A "think with 2D first, drive with dedicated 3D priors" chain: the VLM autoregressively grounds planning-relevant boxes, region-level features are sampled from a frozen geometric foundation model at those boxes, projected into LLM space, and interleaved after the grounding output so trajectory decoding conditions on localized geometry; supervised by a new planning-relevant grounding task.

Key results

• On nuScenes valset, GeoCoTDrive achieves 0.11 average collision rate versus 0.18 for the best prior VLA (LaST-VLA) and 0.22 for SpaceDrive.
• On NAVSIM navtest, GeoCoTDrive reaches 87.6 PDMS over a 85.9 Qwen2.5-VL-3B baseline and 87.3 for LaST-VLA (InternVL3-8B).
• On Bench2Drive, GeoCoTDrive reaches 78.68 DS and 55.42 SR, above ORION (77.74 DS) and SpaceDrive (78.02 DS).

Surprising findings

• Directly interleaving global geometry tokens yields almost no gain over the baseline (PDMS 85.8 vs 85.9), while localized retrieval gives 87.6.

Limitations

• Relies on front-view images only, without multi-view or temporal modeling.
• Localized geometry retrieval depends on grounding accuracy; incorrect regions yield less relevant geometry.
• Gains on NAVSIM are modest (85.9 to 87.6 PDMS).

Experimental setup

Datasets: nuScenes NAVSIM Bench2Drive OmniDrive NAVSIM-Traj B2D-Chat PlanningGrounding

Benchmarks: nuScenes NAVSIM navtest Bench2Drive PlanningGrounding test set

Metrics: L2 collision rate intersection rate PDMS NC DAC TTC EP Comfort driving score

Baselines: ST-P3 UniAD VAD-Base AD-MLP BEV-Planner++ UAD RDA-Driver OmniDrive ORION VGGT-Drive LaST-VLA SpaceDrive

Builds on: OmniDrive ORION VGGT Depth Anything V3 LLaVA-1.5

Closest papers by meaning

• LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.93)
• SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving (CVPR 2026 · similarity 0.92)
• Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning (AAAI 2026 · similarity 0.92)
• $AutoDrive\text{-}P^3$: Unified Chain of Perception–Prediction–Planning Thought via Reinforcement Fine-Tuning (ICLR 2026 · similarity 0.91)
• VGGDrive: Empowering Vision-Language Models with Cross-View Geometric Grounding for Autonomous Driving (CVPR 2026 · similarity 0.91)
• SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Driving (CVPR 2026 · similarity 0.91)