Research map / Vision-language and multimodal / Robot action and navigation

LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving

NeurIPS 2026 · Vision-language and multimodal › Robot action and navigation · approach: Vision-language-action foundation models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LaST-VLA, a driving VLA that reasons in a latent spatio-temporal CoT supervised by Cosmos world-model and VGGT geometry features, trained with progressive SFT and GRPO, reaching 91.3 PDMS on NAVSIM v1 and 87.1 EPDMS on NAVSIM v2.

Problem

Explicit textual chain-of-thought in driving VLAs creates a semantic-perceptual gap and high latency, while unsupervised latent CoT is physics-agnostic and unstable during training.

Approach

Supervise continuous latent CoT tokens by aligning adapter-projected hidden states to frozen features from a video world model (Cosmos, dynamics) and a 3D foundation model (VGGT, geometry), using a two-phase SFT schedule with structured causal masking, then refine with GRPO while adapters are frozen.

Key results

• LaST-VLA-8B reaches 91.3 PDMS on NAVSIM v1 (RL), above Recogdrive-2B at 90.8 and InternVL3-8B RL at 87.2.
• LaST-VLA-8B reaches 87.1 EPDMS on NAVSIM v2, versus 86.1 for DriveVLA-W0-7B.
• On SURDS, LaST-VLA-8B scores 84.29% average and 81.19% Motion on NuDynamics, versus 76.61% and 73.97% for InternVL3-8B.

Surprising findings

• Textual CoT improves PDMS by only 1.2 over no-text baseline, and unsupervised latent CoT reaches 89.8 PDMS, which is above the textual variant.

Limitations

• Single front-view camera causes failures when trajectories extend beyond the field of view at sharp turns or intersections.
• Unsupervised latent CoT is unstable during RL, and supervision is needed for stability.
• Relies on external frozen teacher models (Cosmos, VGGT) during training.

Experimental setup

Datasets: NAVSIM navtrain navtrain-hard-24k SURDS nuScenes NuDynamics

Benchmarks: NAVSIMv1 NAVSIMv2 SURDS NuDynamics nuScenes

Metrics: PDMS EPDMS accuracy L2 error collision rate

Baselines: DiffusionDrive WoTE DriveVLA-W0-7B AdaThinkDrive-8B Recogdrive-2B InternVL3-8B SURDS-3B Qwen2.5-VL-72B-Instruct HydraMDP++ TransFuser UniAD AutoVLA

Builds on: GRPO InternVL3 VGGT Cosmos NAVSIM

Closest papers by meaning

• OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation (NeurIPS 2026 · similarity 0.94)
• Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.93)
• Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning (AAAI 2026 · similarity 0.92)
• AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.92)
• Less Language, More Latents: Annotation-Efficient VLAs for Driving (NeurIPS 2026 · similarity 0.92)
• DriveSpatial: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving (NeurIPS 2026 · similarity 0.92)