Research map / Reinforcement learning / Visual and representation RL

Unified Noise Steering for Efficient Human-Guided VLA Adaptation

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Vision-based robot RL

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UniSteer, which inverts human corrective actions into noise targets for a frozen flow-matching VLA (pi0) so the same noise actor can be trained by both supervised noise regression and RL, adapting a real robot from 20% to 90% success in about 66 minutes.

Problem

Adapting diffusion/flow-matching VLA policies to real-world distributions under a tight on-robot interaction budget; noise-space RL explores inefficiently, while human corrections are given in action space and cannot directly supervise the noise variables.

Approach

Approximate action-to-noise inversion of the frozen flow-matching decoder via per-step fixed-point iteration, turning human corrective action chunks into noise targets that supervise a noise actor jointly optimized with a noise-space critic under RL.

Key results

• UniSteer raises average success rate from 20% to 90% in 66 minutes on four real-world tasks, versus 55% for DSRL and 60% for DAgger.
• On OOD positions, UniSteer reaches 100% on Pick up Spoon, Stack Blocks, and Insert Square, while DSRL gets 0%, 0%, and 25%.
• Fixed-point inversion (M=16) yields action reconstruction MSE of 0.00122 on Pick up Spoon, versus 0.06516 for optimization-based inversion, at lower inversion time.

Surprising findings

• Direct action supervision reached 8/8 on Pick up Spoon, matching fixed-point inversion's success despite much higher training time.
• Only-RL noise-space training achieved 0% OOD success on Pick up Spoon, and RL-then-SFT ordering underperformed SFT-then-RL.

Limitations

• Recovered noise targets may shift from the noise actor's initial distribution.
• Evaluated only on four tasks and one robot platform (AgileX Piper).
• Small evaluation sample size (20 trials per task) and no statistical significance reported.

Experimental setup

Datasets: 30 task demonstrations per task human corrective interventions collected online

Benchmarks: real-world pick up spoon real-world stack blocks real-world insert square real-world fold towel

Metrics: success rate training time (min) action reconstruction MSE

Baselines: DSRL DAgger

Builds on: DSRL DAgger pi0 flow matching

Closest papers by meaning

• PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.90)
• ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models (NeurIPS 2026 · similarity 0.89)
• VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Models (NeurIPS 2026 · similarity 0.89)
• ActionUNet: Improving Robustness of VLA Models with Efficient Multi-scale Fine-tuning (NeurIPS 2026 · similarity 0.89)
• NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models (NeurIPS 2026 · similarity 0.89)
• VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model (ICML 2026 · similarity 0.88)