Research map / Multi-agent systems and games / Human-AI and goal-conditioned

Grounding Driving VLA via Inverse Kinematics

NeurIPS 2026 · Multi-agent systems and games › Human-AI and goal-conditioned · approach: Data-driven traffic simulation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Re-formulates Driving VLA trajectory planning as inverse kinematics: the LLM predicts a future visual state and a separate cross-attention diffusion IK Network decodes the trajectory only from current and predicted future visual tokens, yielding a 0.5B model competitive with 7-8B VLAs on NAVSIM and nuScenes.

Problem

Driving VLAs predict trajectories while ignoring visual tokens ("blind planning"), relying on ego status and text commands as shortcuts.

Approach

Add a next visual state prediction objective so the LLM must predict the future BEV/visual state, and decode trajectories with an IK Network conditioned only on the current and predicted future visual states (no ego status or command), using a cross-attention conditional diffusion model.

Key results

• On NAVSIM-v1 the 0.5B model reaches PDMS 92.2 (with GRPO), above OpenDriveVLA* (73.2) and slightly above DriveFine-8B (91.8).
• On NAVSIM-v2 the 0.5B model reaches EPDMS 90.6 versus 89.7 for DriveFine-8B and 70.2 for OpenDriveVLA*.
• On nuScenes the 0.5B model gets ADE 0.06 vs 0.33 for OpenDriveVLA-7B; in counterfactual stitching it decelerates on 63.8% of samples vs 44.2% for OpenDriveVLA-0.5B.

Surprising findings

• Existing Driving VLAs barely change trajectories when a vehicle is stitched into the camera image, indicating blind planning.
• Replacing the IK Network with a direct MLP head raises ADE from 0.06 to 0.31 on nuScenes.

Limitations

• Relies on the nuScenes/NAVSIM benchmarks where the ego-status shortcut is strong; generalization beyond them not shown.
• IK Network uses diffusion sampling with 50 steps, adding inference cost.
• The 'inverse kinematics' framing is stylistic rather than strict vehicle dynamics, as stated by the authors.

Experimental setup

Datasets: NAVSIM navtrain nuScenes nuScenes-QA nu-Caption nu-X

Benchmarks: NAVSIM-v1 NAVSIM-v2 nuScenes val nu-Caption nu-X nuScenes-QA

Metrics: PDMS EPDMS ADE collision rate L2 GradCAM pointing mass average precision IoU@top20 F1@top20 decel ratio

Baselines: OpenDriveVLA* OpenDriveVLA-0.5B DriveFine DriveVLA-W0 ReCogDrive AdaThinkDrive AutoVLA Alpamayo-R1 Impromptu-VLA

Builds on: RecogDrive OpenDriveVLA DDPM GRPO DriveVLA-W0

Closest papers by meaning

• FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving (NeurIPS 2025 · similarity 0.90)
• Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models (NeurIPS 2025 · similarity 0.88)
• OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation (NeurIPS 2026 · similarity 0.88)
• LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving (NeurIPS 2026 · similarity 0.88)
• ChainFlow-VLA: Causal Flow Planning with Vision-Language Models (NeurIPS 2026 · similarity 0.88)
• SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving (CVPR 2026 · similarity 0.88)