Research map / Vision-language and multimodal / Robot action and navigation
BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BridgeVLA, a 3D VLA that projects point clouds into orthographic images and predicts 2D heatmaps for actions on a PaliGemma VLM, pre-trained on heatmap object grounding, reaching SOTA on RLBench, COLOSSEUM, GemBench and real-robot tests.
Problem
Existing 3D VLA models convert actions to token sequences without spatial structure, giving poor sample efficiency and a distribution shift between 3D fine-tuning inputs and 2D VLM pre-training.
Approach
Align input and output in a shared 2D image space: render orthographic projection images as VLM input, predict per-view heatmaps as translational action output, and pre-train the VLM to output text-conditioned object heatmaps on RoboPoint detection data.
Key results
- On RLBench (18 tasks, 100 demos each), BridgeVLA reaches 88.2% average success vs 81.4% for RVT-2.
- On COLOSSEUM, BridgeVLA averages 64.0% vs 56.7% for RVT-2.
- On real robot basic setting with 10 trajectories per task, BridgeVLA achieves 96.9% average vs 90% for RVT-2 and 28.5% for SpatialVLA (50 trajectories).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 2D heatmap prediction instead of direct position regression (ablation) | BridgeVLA · RLBench (18 tasks) | average success rate (%): 31.4% to 88.2% | Helped |
| Keep per-pixel 3D position features out of the VLM image features (ablation) | BridgeVLA · RLBench (18 tasks) | average success rate (%): 56.2% to 88.2% | Helped |
| Add 2D heatmap pre-training to the VLM backbone (ablation) | BridgeVLA · Real-robot language-related generalization settings | success rate | Helped |
| Use BridgeVLA instead of RVT-2 | 3D manipulation policy · RLBench (18 tasks) | average success rate (%): 81.4% to 88.2% | Helped |
| Use BridgeVLA instead of RVT-2 | 3D manipulation policy · COLOSSEUM | average success rate (%): 56.7% to 64.0% | Helped |
| Use BridgeVLA instead of 3D-LOTUS++ | 3D manipulation policy · GemBench | average success rate: 48.0 to 50.0 | Helped |
| Train BridgeVLA with 3 trajectories per task rather than 10 | BridgeVLA · Real-robot Basic | success rate: 95.4% | No clear effect |
| Use BridgeVLA instead of RVT-2 | Real-robot manipulation policy · Real-robot Basic | success rate: 90% to 96.9% | Helped |
Surprising findings
- Adding explicit per-pixel 3D position inputs to the VLM dropped RLBench performance from 88.2% to 56.2%, suggesting distribution shift from pre-training hurts.
- Standard VLAs (pi0, SpatialVLA with 10 trajectories) fail almost completely in the low-data real-robot regime, while 3D heatmap methods remain strong.
Limitations
- Lower performance on Place Cups, attributed to keypoints occluded in all projection views.
- Near-zero success on GemBench L4 long-horizon tasks.
- Real-robot Category generalization remains low in absolute terms; pre-training data is third-person and object-only.
Experimental setup
Datasets: RoboPoint (120K object detection split) RLBench COLOSSEUM GemBench
Benchmarks: RLBench COLOSSEUM GemBench Real-robot (Franka Research 3)
Metrics: average success rate average rank success rate
Baselines: Image-BC (CNN) Image-BC (ViT) C2F-ARM-BC HiveFormer PolarNet PerAct Act3D 3D Diffuser Actor RVT RVT-2 R3M-MLP MVP-MLP
Builds on: RVT-2 RVT PaliGemma PerAct RoboPoint
Closest papers by meaning
- CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models (CVPR 2025 · similarity 0.91)
- HAMSTER: Hierarchical Action Models for Open-World Robot Manipulation (ICLR 2025 · similarity 0.90)
- PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
- VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
- Beyond Data Scaling: Representation-Centric Pre-training for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
- NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.