Research map / Vision-language and multimodal / Multimodal LLMs
NuPlanQA: A Large-Scale Dataset and Benchmark for Multi-View Driving Scene Understanding in Multi-Modal Large Language Models
Sung-Yeon Park, Can Cui, Yunsheng Ma, Ahmadreza Moradipari, Rohit Gupta, Kyungtae Han, Ziran Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add BEV-Fusion module (multi-view, multi-frame) (ablation) | LLaMA-3.2-Vision-11B baseline · NuPlanQA-Eval | accuracy total: 65.5 -> 78.7 | Improved |
| add BEV-Fusion module (multi-view, multi-frame) (ablation) | LLaMA-3.2-Vision-11B baseline · NuPlanQA-Eval spatial relations | accuracy: 60.8 -> 74.1 | Improved |
| add BEV-Fusion module (multi-view, multi-frame) (ablation) | LLaMA-3.2-Vision-11B baseline · NuPlanQA-Eval ego-centric reasoning | accuracy: 67.0 -> 81.9 | Improved |
| add BEV-Fusion module (multi-view, multi-frame) (ablation) | LLaMA-3.2-Vision-11B baseline · NuPlanQA-Eval road env. perception | accuracy: 68.7 -> 80.0 | Improved |
| add BEV-Fusion with single-view single-frame input (ablation) | LLaMA-3.2-Vision-11B baseline · NuPlanQA-Eval | accuracy total: 65.5 -> 71.4 | Improved |
| add multi-view images (versus front view only) with BEV-Fusion, single-frame (ablation) | BEV-LLM · NuPlanQA-Eval | accuracy total: 71.4 -> 76.2 | Improved |
| add historical frames to single-view BEV-Fusion (ablation) | BEV-LLM · NuPlanQA-Eval | accuracy total: 71.4 -> 72.4 | Improved |
| add historical frames to multi-view BEV-Fusion (ablation) | BEV-LLM · NuPlanQA-Eval | accuracy total: 76.2 -> 78.7 | Improved |
| use multi-frame instead of single-frame input for GPT-4o | GPT-4o · NuPlanQA-Eval | accuracy total: 81.6 -> 81.5 | No clear effect |
| use multi-frame instead of single-frame input for Gemini-1.5-Pro | Gemini-1.5-Pro · NuPlanQA-Eval | accuracy total: 79.4 -> 78.1 | Hurt |
| use multi-frame instead of single-frame input for LLaVA-OV 7B | LLaVA-OV 7B · NuPlanQA-Eval | accuracy total: 75.8 -> 77.8 | Improved |
| scale LLaVA-NV from 7B to 32B | LLaVA-NV · NuPlanQA-Eval multi-frame | accuracy total: 56.8 -> 71.3 | Improved |
Closest papers by meaning
- Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models (CVPR 2024 · similarity 0.91)
- MAPLM: A Real-World Large-Scale Vision-Language Benchmark for Map and Traffic Scene Understanding (CVPR 2024 · similarity 0.91)
- STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving (NeurIPS 2025 · similarity 0.90)
- Embodied Scene Understanding for Vision Language Models via MetaVQA (CVPR 2025 · similarity 0.89)
- DriveSpatial: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving (NeurIPS 2026 · similarity 0.89)
- NuScenes-QA: A Multi-Modal Visual Question Answering Benchmark for Autonomous Driving Scenario (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
