Research map / Vision-language and multimodal / Robot action and navigation
VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models
Borong Zhang, Jiahao Li, Jiachen Shen, Yuhao Zhang, Yishuai Cai, Yuanpei Chen, Juntao Dai, Jiaming Ji, Yaodong Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add L1 data to L0 training (ablation) | pi0 · StaticObstacles L1 | success rate: 0.36 -> 0.90 | Improved |
| add L1 data to L0 training (ablation) | pi0 · UnseenObjects L1 | success rate: 0.64 -> 0.98 | Improved |
| add L1 data to L0 training (ablation) | pi0 · UnseenObjects L2 | success rate: 0.16 -> 0.02 | Hurt |
| add L1 data to L0 training (ablation) | pi0 · DynamicDistractors L2 | success rate: 0.16 -> 0.32 | Improved |
| add L1 data to L0 training (ablation) | pi0 · StaticObstacles L2 | success rate: 0.38 -> 0.40 | No clear effect |
| train on all L0 tasks instead of three suites (ablation) | pi0 · UnseenObjects L1 | success rate: 0.64 -> 0.52 | Hurt |
| move from L0 to L2 difficulty | pi0 real robot · Franka Research 3 average success | success rate (%): 60.0% -> 3.3% | Hurt |
| add ambient light and camera pose shift | pi0 real robot · Franka L0 tasks | average success rate (%): 60.0% -> 20.0% | Hurt |
| move from V3 to V4 perturbation | VLAs vs Qwen3-VL-8B · UnseenObjects | performance drop (%): VLAs 30.5% -> 50.5%; VLM 3.3% -> 6.7% | Hurt |
| invalidate instructions | VLAs on LIBERO · LIBERO | performance drop: 28% drop | Hurt |
| invalidate instructions | VLAs on VLA-Arena · VLA-Arena | performance drop: 52-64% collapse from 79% baseline | Hurt |
Closest papers by meaning
- VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Models (NeurIPS 2026 · similarity 0.93)
- Beyond Data Scaling: Representation-Centric Pre-training for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.92)
- Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models (NeurIPS 2026 · similarity 0.92)
- ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models (NeurIPS 2026 · similarity 0.92)
- VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Models (NeurIPS 2025 · similarity 0.92)
- SAFE: Multitask Failure Detection for Vision-Language-Action Models (NeurIPS 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
