Research map / Vision-language and multimodal / CLIP and zero-shot

Reinforcing VLAs in Task-Agnostic World Models

NeurIPS 2026 · Vision-language and multimodal › CLIP and zero-shot · approach: Adapting VLMs for zero-shot classification · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RAW-Dream, which post-trains VLAs with GRPO inside a task-agnostic action-conditioned video world model (Wan2.1-based) using zero-shot Qwen3-VL success rewards and a dual-noise verification filter for hallucinated successes.

Problem

Existing world-model-based RL for VLA post-training requires target-task rollouts to train the world model and task-specific reward models, which limits scaling to unseen tasks.

Approach

Decouple dynamics from task semantics: a world model pre-trained on task-free play/exploration data plus an off-the-shelf VLM reward enables zero-shot RL on new tasks, with dual-noise verification (replaying successful rollouts under fresh diffusion noise and requiring VLM agreement) to filter world-model hallucinations.</dingadjust>

Key results

• On LIBERO, zero-shot-WM RL with Qwen3-VL reward lifts 1-shot SFT average success from 43.4% to 52.3% (+8.9), above online RL with 522 real episodes (47.9%).
• Co-Train WM RL reaches 57.1% average, close to online RL with 2,570 episodes (58.8%).
• On a real AgileX Piper arm, RL from 3-shot SFT improves average success from 50.0% to 71.7% across four tasks.

Surprising findings

• A 1-shot fine-tuned VideoMAE reward model degrades performance below the SFT baseline on all suites, while zero-shot Qwen3-VL performs better.
• Zero-shot world model RL on LIBERO-Long outperforms long-horizon online RL (35.8% vs 22.4%).

Limitations

• Zero-shot world model fidelity is poor on novel-object suites (LIBERO-Object FVD 233), limiting RL gains there.
• VLM reward quality degrades on Goal (precision 61%, recall 25%), capping improvement.
• Relies on 1-shot/3-shot expert demonstrations for SFT anchoring; DNV adds overhead.

Experimental setup

Datasets: LIBERO-90 LIBERO-Spatial LIBERO-Object LIBERO-Goal LIBERO-Long Open X-Embodiment real-world play data (~4h)

Benchmarks: LIBERO-Spatial LIBERO-Object LIBERO-Goal LIBERO-Long real-world AgileX Piper tasks

Metrics: success rate PSNR SSIM LPIPS FVD precision recall F1

Baselines: 1-shot SFT Online RL WoVR WM Robometer 1-shot fine-tuned VideoMAE reward model

Builds on: GRPO WMPO OpenVLA-OFT Wan2.1 WoVR

Closest papers by meaning

• RLVR-World: Training World Models with Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
• Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning (ICLR 2024 · similarity 0.89)
• World–Value–Action Model: Implicit Planning for Vision–Language–Action Systems (NeurIPS 2026 · similarity 0.89)
• Zero-shot World Models via Search in Memory (NeurIPS 2025 · similarity 0.89)
• NoRD: A Data-Efficient Vision-Language-Action Model that Drives without Reasoning (CVPR 2026 · similarity 0.89)
• Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RL (NeurIPS 2026 · similarity 0.89)