Research map / Vision-language and multimodal / Robot action and navigation
VLMbench: A Compositional Benchmark for Vision-and-Language Manipulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add vision and language inputs rather than using either modality alone (ablation) | 6D-CLIPort · VLMbench task categories | success rate | Helped |
| Remove visual input (ablation) | 6D-CLIPort · Drop, Shape Sorter, Pour, and Door | success rate | Hurt |
| Remove high-level language instructions (ablation) | 6D-CLIPort · VLMbench task categories | success rate | Hurt |
| Provide ground-truth positions instead of estimating positions (ablation) | 6D-CLIPort · Pick & Place, seen | success rate: 28.28 -> 83.84 | Helped |
| Provide ground-truth positions instead of estimating positions (ablation) | 6D-CLIPort · Stack, seen | success rate: 22.19 -> 58.93 | Helped |
| Provide ground-truth positions instead of estimating positions (ablation) | 6D-CLIPort · Shape Sorter, seen | success rate: 17.33 -> 18.00 | Helped |
| Provide ground-truth orientations instead of estimating orientations (ablation) | 6D-CLIPort · Pick & Place, seen | success rate: 28.28 -> 28.03 | No clear effect |
| Provide ground-truth orientations instead of estimating orientations (ablation) | 6D-CLIPort · Drop, seen | success rate: 6.42 -> 17.91 | Helped |
| Provide partial ground-truth pose values (ablation) | 6D-CLIPort · Pour and Door | success rate | Mixed |
| Evaluate on unseen rather than seen object combinations | 6D-CLIPort · VLMbench task categories | success rate | Mixed |
| Use novel compositions of variations | 6D-CLIPort · Shape, Level, Action Type, and Amount variations | success rate | Hurt |
Closest papers by meaning
- Vision-Language Foundation Models as Effective Robot Imitators (ICLR 2024 · similarity 0.90)
- OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints (CVPR 2025 · similarity 0.89)
- Object-Centric Prompt-Driven Vision-Language-Action Model for Robotic Manipulation (CVPR 2025 · similarity 0.89)
- ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation (CVPR 2024 · similarity 0.89)
- EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (ICML 2025 · similarity 0.89)
- RoboProcessBench: Benchmarking Process-Aware Understanding in Visual-Language Robotic Manipulation (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.