Research map / Vision-language and multimodal / Robot action and navigation
Think Densely, Act Sparsely: Latent Expert Cognitive Chains for Vision-Language-Action Autonomous Driving
Jie Wang, Guang Li, Zhijian Huang, Jinlong Li, Chenxu Dang, Hangjun Ye, Yahong Han, Long Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes latent expert cognitive chains for vision-language-action autonomous driving, aiming to think densely in latent space while acting with sparse trajectory outputs.
Problem
Existing VLA driving models are trained with sparse supervision (language instructions or trajectories), forcing compression or loss of dense visual, semantic, and geometric scene information needed for driving.
Approach
Introduces latent expert cognitive chains that let the model reason densely over visual, semantic, and geometric information in latent form before producing sparse driving actions, instead of mapping dense observations directly to sparse outputs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the full latent expert cognitive chain (DINOv3, SAM 3, DepthAnything 3) with hybrid trajectory formulation (ablation) | Qwen2.5-VL-7B baseline · NAVSIM-v1 | PDMS: 86.04 -> 89.81 | Improved |
| replace direct coordinate regression with discrete motion primitive retrieval (ablation) | Qwen2.5-VL-7B baseline · NAVSIM-v1 | PDMS: 86.04 -> 87.44 | Improved |
| add continuous offset refinement on top of motion primitives (ablation) | motion primitive trajectory head · NAVSIM-v1 | PDMS: 87.44 -> 88.49 | Improved |
| add DINOv3 expert latent tokens (ablation) | MT + Offset · NAVSIM-v1 | PDMS: 88.49 -> 89.07 | Improved |
| add SAM 3 semantic expert after DINOv3 (ablation) | MT + Offset + DINOv3 · NAVSIM-v1 | PDMS: 89.07 -> 89.31 | Improved |
| add DepthAnything 3 geometric expert after DINOv3 and SAM 3 (ablation) | MT + Offset + DINOv3 + SAM 3 · NAVSIM-v1 | PDMS: 89.31 -> 89.81 | Improved |
| add task-specific latent tokens instead of using visual tokens with task heads (ablation) | LECDrive · NAVSIM-v1 | PDMS: 88.73 -> 89.81 | Improved |
| aggregate latent embeddings from four VLM layers instead of the final layer only (ablation) | LECDrive · NAVSIM-v1 | PDMS: 89.26 -> 89.81 | Improved |
| use 8 latent tokens per expert instead of 2 (ablation) | LECDrive · NAVSIM-v1 | PDMS: 89.13 -> 89.81 | Improved |
| use 7B instead of 3B base VLM (ablation) | LECDrive · NAVSIM-v1 | PDMS: 89.21 -> 89.81 | Improved |
| add all three experts without motion-primitive/offset trajectory formulation (ablation) | Qwen2.5-VL-7B baseline · NAVSIM-v1 | PDMS: 86.04 -> 89.15 | Improved |
| change expert order from DINO-SAM-DEPTH to DINO-DEPTH-SAM or SAM-DINO-DEPTH (ablation) | LECDrive · NAVSIM-v1 | PDMS: 89.81 -> 89.18 and 89.66 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 3 of 3 on most shared results.
- VGGDrive — VGGDrive: Empowering Vision-Language Models with Cross-View Geometric Grounding for Autonomous Driving (CVPR 2026) · better on 6 of 8 shared results
- ReCogDrive — ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving (ICLR 2026) · better on 1 of 1 shared result
- DriveVLA-W0 — DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving (ICLR 2026) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.92)
- SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Driving (CVPR 2026 · similarity 0.91)
- Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.91)
- DriveSpatial: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving (NeurIPS 2026 · similarity 0.91)
- OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation (NeurIPS 2026 · similarity 0.90)
- Less Language, More Latents: Annotation-Efficient VLAs for Driving (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
