Research map / Vision-language and multimodal / Multimodal LLMs

MindDriver: Introducing Progressive Multimodal Reasoning for Autonomous Driving

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MindDriver, a VLM that reasons progressively from text semantics to dreamed future scene images to trajectories, trained with feedback-guided data annotation and two-stage GRPO-based progressive RFT.

Problem

Textual chain-of-thought for VLM-based end-to-end driving suffers a semantic-to-physical space gap, while prior image-based reasoning lacks planning-oriented guidance for generating accurate future scenes.

Approach

Chain text reasoning (semantic understanding) into a unified autoregressive model that dreams a future front-view image (discrete VQ tokens) and then predicts the trajectory, with reasoning data annotated by Qwen2.5-VL-72B and filtered/re-annotated via format, decision, and logic filters plus error feedback, then optimized in two GRPO stages (CLIP-similarity image reward, then L2 trajectory reward

Key results

• On nuScenes without ego status, MindDriver reaches 0.53 m average L2 and 0.17% average collision under the ST-P3 metric, and 0.93 m L2 and 0.38% collision under the UniAD metric, outperforming FSDrive (0.55 / 0.21 and 1.01 / 0.44).
• On Bench2Drive, MindDriver achieves 65.48 driving score and 39.55% success rate without ego status, above DriveAdapter (64.22 / 33.08) and ReasonPlan (64.01 / 34.55), but below AutoVLA (78.84 / 57.73), which is trained on more data.
• Future frame generation FID on nuScenes at 128x192 is 9.4, better than FSDrive (10.1) and diffusion-based DriveDreamer and Drive-WM.

Surprising findings

• Pure image CoT gives limited benefit over no-CoT, and the naive raw CoT from Qwen2.5-VL-72B substantially degrades performance (avg L2 2.48 m vs 0.99 m with no CoT) until filtered.
• Dreaming the next 0.5 s frame works better than 1 s or 1.5 s frames, attributed to alignment with the history frame interval.

Limitations

• Highly GPU-dependent and computationally demanding for inference and training.
• Only front-view images are generated.

Experimental setup

Datasets: nuScenes Bench2Drive

Benchmarks: nuScenes Bench2Drive

Metrics: L2 error collision rate FID driving score success rate efficiency comfort

Baselines: FSDrive AutoVLA ReasonPlan UniAD VAD OccWorld DriveAdapter RDA-Driver

Builds on: GRPO FSDrive ReasonPlan AutoVLA MoVQGAN

Closest papers by meaning

• Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning (AAAI 2026 · similarity 0.91)
• $AutoDrive\text{-}P^3$: Unified Chain of Perception–Prediction–Planning Thought via Reinforcement Fine-Tuning (ICLR 2026 · similarity 0.90)
• DriveCombo: Benchmarking Compositional Traffic Rule Reasoning in Autonomous Driving (CVPR 2026 · similarity 0.90)
• DriveSpatial: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving (NeurIPS 2026 · similarity 0.90)
• Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving (NeurIPS 2026 · similarity 0.90)
• LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.90)