Research map / Vision-language and multimodal / Multimodal LLMs
Beyond Description: Cognitively Benchmarking Fine-Grained Action for Embodied Agents
Dayong Liu, Chao Xu, Weihong Chen, Suyu Zhang, Juncheng Wang, Jiankang Deng, Baigui Sun, Yang Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| SFT on CFG-Bench data | Qwen2.5-VL-7B · EB-ALFRED | success rate: 4.7 -> 9.7 | Improved |
| SFT on CFG-Bench data | Qwen2.5-VL-7B · EB-Manipulation | success rate: 9.6 -> 15.3 | Improved |
| SFT on CFG-Bench data | InternVL3-8B · EB-ALFRED | success rate: 10.3 -> 14.0 | Improved |
| SFT on CFG-Bench data | InternVL3-8B · EB-Manipulation | success rate: 11.5 -> 15.8 | Improved |
| SFT on FAVOR data instead of CFG data | Qwen2.5-VL-7B · EB-ALFRED | success rate: 4.7 -> 4.3 | Hurt |
| SFT on FAVOR data | Qwen2.5-VL-7B · EB-Manipulation | success rate: 9.6 -> 8.2 | Hurt |
| add CFG-Bench SFT before VLA training | StarVLA-pi (Qwen3-VL-4B) · LIBERO Goal | success rate: 98.6 -> 99.4 | Improved |
| add CFG-Bench SFT before VLA training | StarVLA-pi (Qwen3-VL-4B) · LIBERO Long | success rate: 94.4 -> 95.2 | Improved |
| add CFG-Bench SFT before VLA training | StarVLA-pi (Qwen3-VL-4B) · LIBERO Spatial | success rate: 98.2 -> 98.2 | No clear effect |
| increase input frame rate from 4 to 8 fps | Qwen2.5-VL-7B · CFG-Bench close-ended | accuracy (Avg_c): 40.71 -> 42.40 | No clear effect |
| scale model size from 7B to 72B | Qwen2.5-VL · CFG-Bench close-ended | accuracy (Avg_c): 40.71 -> 51.28 | Improved |
| embodied fine-tuning (RoboBrain2.0 from Qwen2.5-VL-7B) | Qwen2.5-VL-7B · CFG-Bench close-ended | accuracy (Avg_c): 40.71 -> 44.53 | Improved |
Closest papers by meaning
- EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (ICML 2025 · similarity 0.94)
- Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making (NeurIPS 2024 · similarity 0.91)
- BEAR: Dissecting Embodied Abilities in Multimodal Language Models through Skill-level Evaluation and Diagnosis (ICML 2026 · similarity 0.91)
- PhysVLM-AVR: Active Visual Reasoning for Multimodal Large Language Models in Physical Environments (NeurIPS 2025 · similarity 0.90)
- Grounding Multimodal Large Language Models in Actions (NeurIPS 2024 · similarity 0.90)
- Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
