Research map / Vision-language and multimodal / CLIP and zero-shot
Interleave-VLA: Enhancing Robot Manipulation with Image-Text Interleaved Instructions
Cunxin Fan, Xiaosong Jia, Yihang Sun, Yixiao Wang, Jianglan Wei, Ziyang Gong, Xiangyu Zhao, Masayoshi Tomizuka, Xue Yang, Junchi Yan, Mingyu Ding
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train and test with interleaved image-text instructions instead of text | π0 · SimplerEnv Novel Object | success rate: 26.0 ± 3.6 -> 53.8 ± 1.5 | Improved |
| train and test with interleaved image-text instructions instead of text | π0 · SimplerEnv Novel Category | success rate: 19.3 ± 1.5 -> 57.3 ± 2.8 | Improved |
| train with interleaved data but test with text only (ablation) | π0 · SimplerEnv Novel Object | success rate: 26.0 ± 3.6 -> 35.8 ± 0.2 | Improved |
| interleaved instructions on familiar tasks | π0 · SimplerEnv In-Domain | success rate: 68.1 ± 1.3 -> 70.5 ± 1.3 | No clear effect |
| pretrain on Interleaved X-Embodiment dataset (ablation) | Interleave-VLA · Real robot FANUC Out-of-Domain | success rate: 0 -> 71 | Improved |
| interleaved instructions instead of text, both pretrained | π0 · Real robot FANUC Out-of-Domain | success rate: 13 -> 71 | Improved |
| mix Internet and task-specific prompt images (ablation) | Interleave-VLA · SimplerEnv In-Domain | success rate: 59.2 -> 71.0 (vs Internet only) | Improved |
| mix Internet and task-specific prompt images (ablation) | Interleave-VLA · SimplerEnv Out-of-Domain | success rate: 67.1 -> 71.7 (vs task-specific only) | Improved |
| add interleaved format text to visual goal images (ablation) | Interleave-VLA · SimplerEnv Move Near | success rate: 0.0 -> 68.8 | Improved |
| add visual goal image content to text instruction (ablation) | π0 · SimplerEnv Unseen Cat. | success rate: 21.0 -> 54.2 | Improved |
| apply Interleave-VLA to OpenVLA with InternVL2.5 | OpenVLA · VIMA-Bench L1 | success rate: 53.71 -> 83.14 | Improved |
| swap OpenVLA VLM backbone Prismatic to InternVL2.5 (text input) (ablation) | OpenVLA · VIMA-Bench L1-L3 | success rate: 53.71 -> 45.29 (L1), 23.00 -> 24.71 (L2), 23.86 -> 29.71 (L3) | No clear effect |
Closest papers by meaning
- Vision-Language-Action Instruction Tuning: From Understanding to Manipulation (ICLR 2026 · similarity 0.92)
- Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning (ICML 2024 · similarity 0.90)
- Exploring the Limits of Vision-Language-Action Manipulation in Cross-task Generalization (NeurIPS 2025 · similarity 0.90)
- Beyond Data Scaling: Representation-Centric Pre-training for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
- FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies (NeurIPS 2026 · similarity 0.90)
- NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
