Research map / Vision-language and multimodal / Multimodal LLMs

nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Driving

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces nuReasoning, a 20K-clip long-tail driving dataset with spatial, decision, and counterfactual reasoning annotations plus a reasoning and planning benchmark, and a VLA baseline nuVLA showing reasoning supervision improves planning.

Problem

Existing autonomous driving datasets and benchmarks target perception, prediction, or planning and offer limited supervision and evaluation for reasoning over long-tail driving scenes.

Approach

A large real-world long-tail driving dataset with VLM-generated and human-verified spatial, decision, and counterfactual reasoning annotations, paired with both reasoning-QA and planning evaluation, and a VLA baseline (nuVLA) whose reasoning supervision improves trajectory planning even without reasoning at inference.

Key results

• Fine-tuning Qwen3-VL-8B raises geometry choice accuracy from 41.2% to 92.0% and counterfactual choice accuracy from 44.7% to 81.6%.
• nuVLA with spatial+decision+counterfactual reasoning supervision reaches NPS 73.09 and ADE 1.555, versus 64.98 NPS and 1.937 ADE for planning-only nuVLA.
• Zero-shot Alpamayo-1.5 scores NPS 50.45, well below the nuVLA variants, indicating a domain gap.

Surprising findings

• Trajectory hit rate stays at or near 0% for all models even after fine-tuning, while coordinate grounding improves substantially.
• Cosmos-Reason2-8B, pretrained for physical and embodied reasoning, performs close to its base Qwen3-VL-8B and does not improve grounding.

Limitations

• Data collected from a finite set of cities and driving conditions, so coverage of long-tail scenarios is incomplete.
• Benchmark is open-loop and may not reflect closed-loop performance.
• Planning metrics are computed in open-loop evaluation on the test set; trajectory hit rate near zero indicates unsolved future motion reasoning.

Experimental setup

Datasets: nuReasoning training split (17K clips) nuReasoning QA pairs (167K subsampled)

Benchmarks: nuReasoning reasoning benchmark nuReasoning planning benchmark

Metrics: choice accuracy numerical accuracy coordinate hit rate coordinate L2 trajectory hit rate trajectory L2 Token-level F1 NC DA EP

Baselines: UniAD DiffusionDrive AutoVLA SpanVLA Alpamayo-1.5 Gemini-3.1-Pro Gemini-3-Flash Qwen3-VL-8B Cosmos-Reason2-8B Qwen3.5-4B Qwen3.5-9B

Builds on: nuScenes nuPlan NAVSIM DriveLM DiffusionDrive AutoVLA

Closest papers by meaning

• DriveSpatial: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving (NeurIPS 2026 · similarity 0.90)
• Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning (AAAI 2026 · similarity 0.90)
• DriveCombo: Benchmarking Compositional Traffic Rule Reasoning in Autonomous Driving (CVPR 2026 · similarity 0.90)
• AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.89)
• SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models (ICML 2025 · similarity 0.88)
• OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation (NeurIPS 2026 · similarity 0.88)