Research map / Vision-language and multimodal / Multimodal LLMs
Attention to Threat-Relevant Objects: Reasoning Detection in Autonomous Driving via Multimodal Large Language Models
Yulin He, Wei Chen, Xinbiao Gan, Siqi Wang, Haotian Wang, Yusong Tan
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces threat-oriented reasoning detection for autonomous driving, localizing threat objects and reasoning about their threat levels from a driver-centric perspective using multimodal LLMs.
Problem
Traditional autonomous driving detection is class-based and does not identify which objects pose real potential risk to the driver; the paper targets localizing threat objects and reasoning about their threat levels.
Approach
Shifts from class-based detection to threat-oriented reasoning detection, framing it as a task solved with multimodal large language models from a driver-centric perspective.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ThreatCoT (image-level + object-level reasoning with expert toolchain) (ablation) | Qwen2.5VL-7B · Threat-ReasonDet | Mean CRecall: 54.7 -> 59.1 | Improved |
| add ThreatCoT (ablation) | InternVL3-8B · Threat-ReasonDet | Mean CRecall: 63.5 -> 66.5 | Improved |
| add ThreatCoT (ablation) | InternVL3-8B · Threat-ReasonDet | Mean L1: 0.84 -> 0.69 | Improved |
| add image-level information only to ThreatCoT (ablation) | InternVL3-8B · Threat-ReasonDet | M-CRecall / M-L1: 63.5 -> 62.8 recall; 0.84 -> 0.80 L1 | Mixed |
| add object-level cues (class, depth) on top of image-level info (ablation) | InternVL3-8B · Threat-ReasonDet | M-CRecall: 62.8 -> 66.5 | Improved |
| add format reward (ablation) | VisionReasoner-7B (GRPO) · Threat-ReasonDet | M-CRecall: 53.9 -> 58.3 | Improved |
| add non-repeat reward on top of format reward (ablation) | VisionReasoner-7B (GRPO) · Threat-ReasonDet | M-CRecall: 58.3 -> 51.8 | Hurt |
| add prediction number reward on top of format + non-repeat (ablation) | VisionReasoner-7B (GRPO) · Threat-ReasonDet | M-CRecall: 51.8 -> 63.6 | Improved |
| add prediction number reward (ablation) | VisionReasoner-7B (GRPO) · Threat-ReasonDet | M-L1: 1.06 -> 0.78 | Improved |
| ThreatReasoner (GRPO) vs VisionReasoner | end-to-end MLLM · Threat-ReasonDet | Mean CRecall: 53.9 -> 63.6 | Improved |
Closest papers by meaning
- SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models (ICML 2025 · similarity 0.88)
- DriveCombo: Benchmarking Compositional Traffic Rule Reasoning in Autonomous Driving (CVPR 2026 · similarity 0.87)
- DriveAgent-R1: Advancing VLM-based Autonomous Driving with Active Perception and Hybrid Thinking (ICLR 2026 · similarity 0.86)
- Hybrid-Driving: An Autonomous Driving Decision Framework Integrating Large Language Models, Knowledge Graphs and Driving Rules (AAAI 2025 · similarity 0.86)
- World Knowledge-Enhanced Reasoning Using Instruction-Guided Interactor in Autonomous Driving (AAAI 2025 · similarity 0.86)
- nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Driving (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
