AI Paper Atlas

Research map / Large language models / Agents, code and math

Automotive-ENV: Benchmarking Multimodal Models in Automotive Cockpit Environments

Junfeng Yan, Biao Wu, Meng Fang, Ling Chen

NeurIPS 2026 · Large language models › Agents, code and math · approach: LLM-driven embodied and driving agents · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Automotive-ENV, a benchmark for evaluating multimodal models on in-vehicle cockpit GUI tasks involving vehicle-state reasoning, implicit intent, and safety-critical decisions.

Problem

Multimodal models are well studied in web, desktop, and mobile GUI environments, but their performance in automotive cockpit systems is largely unexplored, with unique challenges such as vehicle-state reasoning, implicit intent understanding, and safety-critical decision making.

Approach

A benchmark environment specifically for automotive cockpit GUIs that targets vehicle-state reasoning, implicit intent understanding, and safety-critical decisions beyond conventional GUI interaction settings.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use a larger general-purpose multimodal model instead of GUI-specialized VLMsAutomotive-ENV agents · General Explicitsuccess rate (%): Qwen3-VL-8B 43.7 -> Kimi-K2.5 75.8Improved
move from General to Safety-Critical tasksKimi-K2.5 · Explicit taskssuccess rate (%): 75.8 -> 29.7Hurt
move from explicit to implicit instructionsKimi-K2.5 · General taskssuccess rate (%): 75.8 -> 56.3Hurt
move from explicit to implicit instructionsQwen3.5-122B-A10B · General taskssuccess rate (%): 70.5 -> 52.2Hurt
move from explicit to implicit instructionsGemini 3 Flash · General taskssuccess rate (%): 45.6 -> 47.1No clear effect
GUI-specialized training on web, desktop or mobile dataGUI-specialized VLMs · Automotive-ENVsuccess rate (%)Hurt
increase model size from 3B to 7BQwen2.5-VL · General Explicitsuccess rate (%): 13.6 -> 26.5Improved
increase model size from 3B to 7BJEDI · General Explicitsuccess rate (%): 16.7 -> 25.2No clear effect
increase model size from 2B to 8BMAI-UI · General Implicitsuccess rate (%): 14.2 -> 32.4Improved
increase task difficulty from Easy to HardKimi-K2.5, Qwen3.5-122B-A10B, Gemini 3 Flash · Automotive-ENV difficulty groupssuccess rateHurt
add vehicle-state querying and safety-aware reasoning to literal instruction followingevaluated agents · Safety-Critical Implicitsuccess rate (%)Improved

Experimental setup

Benchmarks: Automotive-ENV

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy