Research map / Vision-language and multimodal / Robot action and navigation
NestedVLA: Learning to Consolidate and Generate Skills for Vision-Language-Action Model
Guodong DU, Wanyu Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NestedVLA, a paradigm for consolidating robot skills learned across heterogeneous tasks, environments, and embodiments in a vision-language-action model, as an alternative to joint multi-task training and model merging.
Problem
Consolidating skills learned across heterogeneous robotic tasks, environments, and embodiments into one VLA model is hard: data is fragmented, joint multi-task training suffers interference, and model merging causes parameter conflicts or needs architectural changes.
Approach
Advocates a paradigm in which skills are learned and consolidated as nested modules within a VLA, rather than via joint training or parameter merging; details unavailable from the abstract provided.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| generate action-expert update with a hypernetwork instead of static or dynamic merging | NestedVLA · MetaWorld, ManiSkill, CALVIN average | success rate: 25.6 -> 66.7 | Improved |
| add per-task residual storage to merging (dynamic merging) at 2.09x-2.89x storage | model merging baselines · average success | success rate: 14.8 (Averaging) -> 22.6-25.6 | Improved |
| add sign-conflict resolution or singular-vector compression to averaging | static merging (TIES, WUDI, TSV) · average success | success rate: 14.8 -> 13.1 / 11.8 / 13.4 | Hurt |
| add per-task output heads (multi-head path) to hypernetwork (ablation) | NestedVLA hypernetwork · MetaWorld/ManiSkill/CALVIN | in-mask standardized loss: MW 1.33 -> 1.08 | Improved |
| add per-task patch embeddings (ablation) | NestedVLA hypernetwork · MetaWorld/ManiSkill/CALVIN | in-mask standardized loss: MW 1.02 (without) vs 1.08 (default) | No clear effect |
| add shared-FC LoRA path (ablation) | NestedVLA hypernetwork · MetaWorld/ManiSkill/CALVIN | in-mask standardized loss: MW 0.91 (without) vs 1.08 (default) | Hurt |
| narrow hypernetwork hidden width from 2048 to 1024 (ablation) | NestedVLA hypernetwork · MetaWorld | in-mask standardized loss: 1.08 -> 1.43 | Hurt |
| use drop-and-rescale instead of 1-bit quantization to compress task vectors (ablation) | NestedVLA skill compression · per-task fidelity | success penalty vs FlowSDE: -71 to -81 pp -> -3 to -5 pp | Improved |
| add rank-r SVD residual to drop-and-rescale (ablation) | NestedVLA skill compression · per-task fidelity | reconstruction error: at most +0.005 fidelity | No clear effect |
| disable out-of-mask penalty (w_out = 0) (ablation) | NestedVLA training loss · average success | success rate: +2.4 pp | Improved |
| add magnitude weighting or absolute loss space alone (ablation) | NestedVLA training loss · average success | success rate | Hurt |
| raise keep ratio K from 10% to 20% (ablation) | skill compression ideal · MetaWorld | success rate: 48.4 -> 60.2 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- EMR-Merging — EMR-Merging: Tuning-Free High-Performance Model Merging (NeurIPS 2024) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agent (CVPR 2026 · similarity 0.93)
- PRISM: Primitive Routing via In-context Skill Mixing for Lifelong VLA (NeurIPS 2026 · similarity 0.92)
- ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
- PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
- VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
- NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
