Research map / Vision-language and multimodal / Multimodal LLMs

BEAR: Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces BEAR, a 4,469-sample interleaved image-video-text benchmark decomposing embodied abilities into 14 atomic skills for skill-level diagnosis of 20 MLLMs, plus a tool-augmented BEAR-Agent that boosts GPT-5 and InternVL3-14B and improves simulated and real-robot manipulation.

Problem

Existing embodied benchmarks evaluate task-level success and cannot attribute failures to specific perceptual or reasoning capabilities, so they give little actionable guidance for improving embodied MLLMs.

Approach

Decompose embodied tasks into 14 atomic skills across 6 categories for isolated skill-level evaluation, use horizontal, vertical, and cross-skill failure attribution to locate bottlenecks (mainly perception and unstable spatiotemporal modeling), and act on these insights with a tool-augmented agent that supplies visual grounding (GroundingDINO, depth, scene graphs) to the MLLM.

Key results

• GPT-5 is the strongest model on BEAR at 52.17% average versus 89.40% human performance; best open-source InternVL3-14B reaches 33.93%.
• Perceptual errors account for 54.8% of limitations versus 22.4% reasoning errors in the cross-skill attribution.
• BEAR-Agent yields a 9.12-point average gain for GPT-5 (17.5% relative) and about 20.17% average improvement when integrated with MOKA in simulation.

Surprising findings

• Chain-of-thought prompting gives mixed and often negligible gains across models, and can degrade performance in some cases.
• Most models score only 20-40% overall, with many open-source models near random on several skills.

Limitations

• BEAR-Agent is developed and evaluated within a limited scope, with no training of the underlying MLLMs.
• Real-robot evaluation is limited to grasping with a single UR5 setup and a small number of instructions.
• The paper does not evaluate training-based improvements such as SFT or RL on the diagnosed bottlenecks.

Experimental setup

Datasets: Ego4D Epic-Kitchens OpenImages V7 PartImageNet AGD20K Open-X-Embodiment ScanNet ScanNet++ ARKitScenes AI2-THOR

Benchmarks: BEAR BEAR-mini

Metrics: success rate IoU episode success rate

Baselines: GPT-5 Gemini-2.5-Pro Claude-4-Sonnet GPT-4o InternVL3-8B InternVL3-14B Qwen2.5-VL-32B-Instruct MOKA

Builds on: MOKA VLMEvalKit EmbodiedBench Chain-of-thought

Closest papers by meaning

• EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (ICML 2025 · similarity 0.92)
• ModularAgent: A Task-Aware Modular Framework for Joint Optimization of Multimodal Large Language Models and World Models (CVPR 2026 · similarity 0.89)
• TPRU: Advancing Temporal and Procedural Understanding in Large Multimodal Models (ICLR 2026 · similarity 0.89)
• NavBench: Probing Multimodal Large Language Models for Embodied Navigation (NeurIPS 2025 · similarity 0.89)
• Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making (NeurIPS 2024 · similarity 0.89)
• An Embodied Generalist Agent in 3D World (ICML 2024 · similarity 0.88)