Research map / Vision-language and multimodal / Robot action and navigation
Beyond Appearance Shifts: Task-Semantic Action Calibration for VLA Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BAS-VLA, a lightweight residual action-calibration layer over a frozen VLA that separates behavior under semantic-breaking edits while staying stable under preserving edits, with an evidence-gated preserving auxiliary.
Problem
VLA policies drift under task-preserving appearance/wording changes and fail to change action sufficiently under semantic-breaking changes (e.g., target-object swaps), so their behavior is neither stable nor task-semantically sensitive.
Approach
A residual action calibrator on a frozen VLA, trained on matched clean/control/break instruction triplets with imitation, consistency, and margin-based separation losses; plus a selective, evidence-gated weak-fusion preserving auxiliary that uses a grounded visual probe activated only under detected nuisance shift.
Key results
- On Milk-Swap, BAS-VLA reaches 98.0% clean, 97.5% control and 0.0% break (clean-task success under break) versus 90.0% for the frozen baseline.
- On Bowl-on-Ramekin style shift, success rises from 42.0% to 70.0% with clean performance unchanged at about 97%.
- On LIBERO-PRO and LIBERO-Plus external protocols, success improves from 36.9% to 71.4% and 54.8% to 77.9%, respectively.
Surprising findings
- The full composition (preserving auxiliary plus breaking core) underperforms the breaking-centered core alone on semantic-break settings (4.0% vs 0.0% break success on Milk-Swap).
- Butter-Swap retains 56.5% old-task success under break, indicating confusion between visually and semantically similar objects.
Limitations
- Preserving auxiliary has a 23.5% false-negative rate on preserving shifts and 7.5% clean false activation.
- Harder swap pairs such as Butter-Swap remain unresolved; executability changes are treated as boundary cases.
- Evaluation is largely simulation-based with few real-robot trials and relies on a single frozen carrier per setting.
Experimental setup
Datasets: LIBERO-Object LIBERO-Spatial LIBERO-10 Bowl-on-Ramekin (constructed) LIBERO-Plus LIBERO-PRO
Benchmarks: LIBERO-Object Milk-Swap LIBERO-Spatial Bowl-on-Ramekin style shift LIBERO-Plus LIBERO-PRO real-robot changed-task tasks
Metrics: task success rate break (clean-task) success control-break gap new/old/other outcome rates Wilson 95% CI
Baselines: SGAC-AC AAC RTC PCD ST4-VLA HAMLET PARR PSSC frozen carrier baseline
Builds on: OpenPI pi0.5 OpenVLA-OFT SAM 2 LIBERO
Closest papers by meaning
- Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
- VLA-Corrector: Lightweight Detect-and-Correct Inference for Adaptive Action Horizon (NeurIPS 2026 · similarity 0.89)
- CofactVLA: Deconfounding Vision-Language-Action Models via Counterfactual Intervention (NeurIPS 2026 · similarity 0.89)
- VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model (ICML 2026 · similarity 0.89)
- VLA-OS: Structuring and Dissecting Planning Representations and Paradigms in Vision-Language-Action Models (NeurIPS 2025 · similarity 0.89)
- VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Models (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.