Research map / Vision-language and multimodal / Robot action and navigation
DiffusionVLA: Scaling Robot Foundation Models via Unified Diffusion and Autoregression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffusionVLA (DiVLA), which unifies a Qwen2-VL autoregressive reasoning VLM with a diffusion policy head via a reasoning injection module, enabling reasoning-aware robot action generation.
Problem
Autoregressive VLA models produce imprecise, slow action outputs from discretized tokens, while diffusion policies lack reasoning ability for complex multi-step robot tasks.
Approach
Combine a pre-trained VLM's autoregressive self-generated reasoning with a diffusion action head; the reasoning output embedding is injected into the policy via FiLM, trained jointly with diffusion loss plus an alpha-weighted next-token loss, and GPT-4o is used to add reasoning annotations to Droid data.
Key results
- Zero-shot bin picking on 102 unseen objects: DiVLA-2B 63.7% vs OpenVLA 28.4%, TinyVLA 23.5%, Octo 19.6%, Diffusion Policy 8.9%.
- Factory sorting average success 66.2% for DiVLA-2B, outperforming OpenVLA (45.3%) by 20.9 points.
- DiVLA-2B runs at 82Hz and DiVLA-7B at 42Hz on an A6000 GPU with vLLM, versus 5Hz for OpenVLA-7B; scaling to 72B raised sorting success to 82.4%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DiVLA-2B instead of the strongest listed baseline, TinyVLA | Multi-task learning · Multi-task learning, in-distribution | success rate (Avg.): 45.5 -> 83.6 | Helped |
| Use DiVLA-2B instead of the strongest listed baseline, TinyVLA | Multi-task learning · Multi-task learning, visual generalization | success rate (Avg.): 28.9 -> 57.8 | Helped |
| Use DiVLA-2B instead of OpenVLA | Robot policy · Zero-shot bin picking, 102 unseen objects | success rate: 28.4 -> 63.7 | Helped |
| Use DiVLA-2B instead of Diffusion Policy | Bimanual table-bussing policy · Table bussing, seen objects | success rate: 45.8 -> 72.9 | Helped |
| Use DiVLA-2B instead of Diffusion Policy | Bimanual table-bussing policy · Table bussing, mixed seen and unseen objects | success rate: 31.2 -> 70.8 | Helped |
| Scale the model from 2B to 7B (ablation) | DiVLA · Factory sorting | success rate: 66.2 -> 74.9 | Helped |
| Scale the model from 2B to 72B (ablation) | DiVLA · Factory sorting | success rate: 66.2 -> 82.4 | Helped |
| Scale the model from 2B to 7B (ablation) | DiVLA · Zero-shot bin picking | success rate: 63.7 -> 66.7 | Helped |
| Scale the model from 2B to 72B (ablation) | DiVLA · Zero-shot bin picking | success rate: 63.7 -> 75.9 | Helped |
| Remove the reasoning injection module (ablation) | DiVLA-2B · Multi-task learning, in-distribution | success rate (Avg.): 83.6 -> 50.3 | Hurt |
| Use one camera view instead of three (ablation) | OpenVLA · Factory sorting | success rate (Avg.): 45.3 -> 12.7 | Hurt |
Surprising findings
- Removing the reasoning injection module dropped multi-task in-distribution average from 83.6% to 50.3%.
- The model was not co-trained with vision-language data yet retained basic VQA ability, though it misidentified some objects.
Limitations
- Visual question answering struggles with object recognition (e.g., toy dragon misidentified as tiger).
- Quantization to 8-bit and 4-bit degrades performance, suggesting need for specialized quantization.
- Performance drops in cluttered scenes; success on bimanual and multi-task settings relies on small trial counts (e.g., 5 to 12 trials).
Experimental setup
Datasets: Droid OXE GPT-4o-generated reasoning annotations on Droid
Benchmarks: Real-robot multi-task learning (Franka) Factory sorting (Franka) Zero-shot bin picking (102 unseen objects) Bimanual table bussing (AgileX)
Metrics: success rate control frequency (Hz)
Baselines: Diffusion Policy Octo OpenVLA TinyVLA
Builds on: Diffusion Policy Qwen2-VL OpenVLA FiLM TinyVLA
Closest papers by meaning
- ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
- DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge (NeurIPS 2025 · similarity 0.89)
- Conditioning Matters: Training Diffusion Policies is Faster Than You Think (NeurIPS 2025 · similarity 0.89)
- ChainFlow-VLA: Causal Flow Planning with Vision-Language Models (NeurIPS 2026 · similarity 0.88)
- TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies (ICLR 2025 · similarity 0.88)
- VLA-Corrector: Lightweight Detect-and-Correct Inference for Adaptive Action Horizon (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.