Research map / Video / Video generation
RoboAlign-R1: Distilled Multimodal Reward Alignment for Robot Video World Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RoboAlign-R1, which distills a fine-tuned Qwen3-VL-8B judge into a lightweight reward model for GRPO post-training of a token-based robot video world model, plus a training-free sliding window re-encoding for long-horizon rollouts.
Problem
Robot video world models are trained with low-level reconstruction or perceptual objectives that are misaligned with instruction following, manipulation success, and physical plausibility, and autoregressive rollouts accumulate errors over long horizons.
Approach
Distill a multimodal teacher judge (six-dimension rubric) into a ~98M student reward for online GRPO post-training, with online iterative re-distillation, and periodically decode-and-re-encode the last predicted frame as fresh context (SWR) to limit drift.
Key results
- RoboAlign-R1 reaches an aggregate RoboAlign-Judge score of 8.52 vs 7.74 for iVideoGPT (+10.1%) on RobotWorldBench.
- SWR with W=6 improves SSIM by 2.8%, PSNR by 0.62 dB, and reduces LPIPS by 9.8% on RT-1 at about 1.1% extra inference time.
- Online iterative distillation raises the aggregate score from 8.09 (one-shot) to 8.52.
Surprising findings
- Closed commercial video models (e.g., Kling 2.6, 6.84) score below embodied world-model baselines like iVideoGPT on the robot rubric.
Limitations
- Evaluation limited to single-arm tabletop and countertop manipulation.
- Downstream policy improvement / control-level gains not validated.
- Main results rely on an in-domain automated judge that the authors built; human study is small-scale.
Experimental setup
Datasets: RT-1 BridgeData V2 CALVIN LIBERO RobotWorldBench
Benchmarks: RobotWorldBench RT-1 BridgeData V2
Metrics: six-dimension RoboAlign-Judge score MSE PSNR SSIM LPIPS ROI-LPIPS inference latency
Baselines: iVideoGPT RLVR-World RoboDreamer Vid2World Wan2.2-TI2V-5B (LoRA) Kling 2.6 Runway Gen-4.5 MiniMax Hailuo 02 Luma Dream Machine HunyuanVideo-I2V LTX-Video Stable Video Diffusion XT
Builds on: GRPO StreamingLLM RLVR-World iVideoGPT Qwen3-VL
Closest papers by meaning
- SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learning (NeurIPS 2026 · similarity 0.91)
- Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in Robotics (NeurIPS 2025 · similarity 0.89)
- RoboDreamer: Learning Compositional World Models for Robot Imagination (ICML 2024 · similarity 0.89)
- Solving New Tasks by Adapting Internet Video Knowledge (ICLR 2025 · similarity 0.88)
- RLVR-World: Training World Models with Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.