Research map / Vision-language and multimodal / Robot action and navigation
Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrations
Chancharik Mitra, Yusen Luo, Raj Saravanan, Dantong Niu, Anirudh Pai, Jesse Thomason, Trevor Darrell, Abrar Anwar, Deva Ramanan, Roei Herzig
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| select task-specific heads via k-NN and finetune only them instead of full-head LoRA | pi0 / pi0.5 · 5 real-robot tasks | success rate: e.g. Press Button Hard 45% -> 75% (pi0) | Improved |
| finetune selected heads vs full LoRA, Press Button Hard on pi0.5 | pi0.5 · Press Button Hard | success rate: 90% -> 85% | Hurt |
| use Robotic Steering instead of full-head LoRA for cost | pi0 · real-robot training | training time / trainable params: 239 min -> 189 min; 1785.9M -> 78.8M | Improved |
| single-task Robotic Steering under environment variation | pi0.5 · Place Marker in Mug perturbations | success rate: lighting 25% -> 47.5%; form 22.5% -> 37.5%; distractor 30% -> 40% | Improved |
| Robotic Steering for generalization to unseen Pick Mug | pi0.5 · Pick Mug (unseen) | success rate: 42.5% -> 65% | Improved |
| multi-task: non-overlapping head selection vs joint LoRA (ablation) | pi0.5 · Place Cube in Bowl | success rate: 37.5% -> 65% | Improved |
| use k-NN regression instead of CMA for head selection (ablation) | Robotic Steering · Place Marker in Mug | success rate: 15% -> 80% | Improved |
| use k-NN regression instead of REINFORCE for head selection (ablation) | Robotic Steering · Place Marker in Mug | head selection time: success 80% vs 80%; time 93 min -> 0.2 min | No clear effect |
| also finetune MLPs of selected blocks (ablation) | Robotic Steering · Place Marker in Mug | success rate: 10% -> 80% | Improved |
| select top 20 heads vs all 144 heads (ablation) | Robotic Steering · Place Marker in Mug | success rate: 45% -> 72.5% | Improved |
| apply Robotic Steering to full finetuning | pi0.5 · Place Marker in Mug / Press Button / Place Cube | success rate: 45% -> 60%; 77.5% -> 87.5%; 55% -> 72.5% | Improved |
| select first token in suffix instead of last token in prefix (ablation) | k-NN head selection · Place Marker in Mug | coefficient of variation: pi0.5: 0.05 -> 0.47; pi0: 0.04 -> 0.12 | Improved |
Closest papers by meaning
- FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies (NeurIPS 2026 · similarity 0.92)
- Beyond Data Scaling: Representation-Centric Pre-training for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.92)
- ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models (NeurIPS 2026 · similarity 0.91)
- PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.91)
- Beyond World-Frame Action Heads: Motion-Centric Action Frames for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.91)
- Exploring the Limits of Vision-Language-Action Manipulation in Cross-task Generalization (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
