Research map / Vision-language and multimodal / Robot action and navigation
LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models
Rongxu Cui, Zongzheng Zhang, Jingrui Pang, Haohan Chi, Jinbang Guo, Saining Zhang, shaoxuan Xie, Xin Jin, Yao Mu, Jiaolong Yang, Guocai Yao, Xianyuan Zhan and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on 500 instead of 50 demonstrations per task (ablation) | pi0.5 · FSHOA L2 | success rate (%): 48.9 -> 51.3 | Improved |
| train on 500 instead of 50 demonstrations per task (ablation) | pi0.5 · FSHOA L2 | collision rate (%): 12.7 -> 10.0 | Improved |
| train on 500 instead of 50 demonstrations per task (ablation) | pi0.5 · FSHOA L2 | LDLJ: -17.78 -> -17.47 | Improved |
| train on 500 instead of 50 demonstrations per task (ablation) | pi0.5 · FSHOA L2 | execution time (s): 362.5 -> 355.0 | Improved |
| perturb object layout (ablation) | pi0.5 · TSA L0 | collision rate (%): 2.7 -> 6.3 | Hurt |
| perturb object layout (ablation) | pi0.5 · TSA L0 | success rate (%): 58.0 -> 56.3 | Hurt |
| introduce unseen objects (ablation) | pi0.5 · TSA L0 | collision rate (%): 2.7 -> 3.3 | No clear effect |
| perturb image noise, init state, view, or scene (ablation) | pi0.5 · TSA L0 | success rate (%): 58.0 -> 56.3-60.7 | No clear effect |
| use reasoning-based RynnBrain-CoP instead of RoboBrain2.0-7B | embodied foundation model · SSR L1 | refusal rate (%): 40.0 -> 72.0 | Improved |
| use reasoning-based RynnBrain-CoP instead of RoboBrain2.0-7B | embodied foundation model · SSR L0 | refusal rate (%): 80.0 -> 56.0 | Hurt |
| use keypose-driven pipeline instead of human teleoperation | data generation · data collection | human effort (min/task): 7.4 -> 1.8 | Improved |
| evaluate on held-out L2 OOD tasks instead of L0 | OpenVLA-OFT · AAG | success rate (%): 50.0 -> 1.3 | Hurt |
Closest papers by meaning
- SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning (NeurIPS 2025 · similarity 0.91)
- VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Models (NeurIPS 2026 · similarity 0.89)
- RedVLA: Physical Red Teaming for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.89)
- SAFE: Multitask Failure Detection for Vision-Language-Action Models (NeurIPS 2025 · similarity 0.88)
- Failing Forward: Adaptive Failure-Informed Learning for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.88)
- AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructions (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
