Research map / Vision-language and multimodal / Multimodal LLMs
RoboTron-Drive: All-in-One Large Multimodal Model for Autonomous Driving
Zhijian Huang, Chengjian Feng, Feng Yan, Baihui Xiao, Zequn Jie, Yujie Zhong, Xiaodan Liang, Lin Ma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add QA augmentation (GPT-4o-mini paraphrasing and multiple-choice conversion) (ablation) | RoboTron-Drive · Six AD datasets (average) | Average: 55.54 -> 58.60 | Improved |
| Add QA augmentation on DriveLM (ablation) | RoboTron-Drive · DriveLM | Average: 47.41 -> 59.67 | Improved |
| Add QA standardization on top of augmentation (ablation) | RoboTron-Drive · NuInstruct | Average: 34.77 -> 42.44 | Improved |
| Add QA standardization on CODA-LM (ablation) | RoboTron-Drive · CODA-LM | Average: 63.93 -> 63.90 | No clear effect |
| Use curriculum learning versus mixed-complexity data from the start (ablation) | RoboTron-Drive · Six AD datasets (average) | Average: 55.71 -> 61.32 | Improved |
| Add more curriculum pre-training stages (S0 to S3) (ablation) | RoboTron-Drive · Six AD datasets (average) | Average: 58.63 -> 61.32 | Improved |
| Train on mixed six datasets instead of each individually (ablation) | RoboTron-Drive · Six AD datasets (average) | Average: 60.12 -> 61.32 | Improved |
| Add perspective-aware prompt (ablation) | RoboTron-Drive · Six AD datasets (average) | Average: 60.69 -> 61.32 | Improved |
| Add perspective-aware prompt on LingoQA (ablation) | RoboTron-Drive · LingoQA | Lingo-Judge: 70.20 -> 69.20 | Hurt |
| Train one generalist on all datasets instead of specialists, then test zero-shot on unseen datasets | RoboTron-Drive · DriveBench (zero-shot) | GPT-Score: 54.48 -> 61.06 | Improved |
| Use AD fine-tuning instead of zero-shot model | RoboTron-Drive-Zero · NuInstruct | Average: 0.00 -> 46.30 | Improved |
| Use RoboTron-Drive instead of Drive-OV (LLaVA-OV with same prompt and data) on LingoQA | RoboTron-Drive · LingoQA | Lingo-Judge: 70.10 -> 69.20 | No clear effect |
Closest papers by meaning
- DriveGPT4-V2: Harnessing Large Language Model Capabilities for Enhanced Closed-Loop Autonomous Driving (CVPR 2025 · similarity 0.89)
- The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset (NeurIPS 2026 · similarity 0.89)
- Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models (CVPR 2024 · similarity 0.88)
- SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models (ICML 2025 · similarity 0.88)
- Generating Multimodal Driving Scenes via Next-Scene Prediction (CVPR 2025 · similarity 0.88)
- Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
