Research map / Video / Video segmentation and tracking
Video Prediction Models as Rewards for Reinforcement Learning
Alejandro Escontrela, Ademi Adeniji, Wilson Yan, Ajay Jain, Xue Bin Peng, Ken Goldberg, Youngwoon Lee, Danijar Hafner, Pieter Abbeel
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add exploration objective (beta from 0 to 1, Plan2Explore reward) to the video-likelihood reward (ablation) | VIPER (VideoGPT-like backbone) · DMC | task return | Improved |
| replace VideoGPT-like video model with MaskGIT (ablation) | VIPER · DMC | task return | Hurt |
| replace MaskGIT with MaskGIT compute cost: up to 8 forward passes per frame instead of 1 (ablation) | VIPER · DMC | reward computation / RL training slowdown: 8x slowdown in reward computation, 2.5x in overall RL training | Hurt |
| replace VideoGPT with recurrent BYOL implicit-density video model (ablation) | VIPER · DMC | task return | Hurt |
| use BYOL instead of MaskGIT as the video model (ablation) | VIPER · DMC | task return | Improved |
| increase video model context length k (ablation) | VIPER (VideoGPT) · DMC | task return | Improved |
| context length k=0 (unconditional frame likelihood) (ablation) | VIPER · DMC | task behavior | Hurt |
| mask the scoreboard when training the video model (ablation) | VIPER · Atari Pong | RL performance / stability | Improved |
| train video model at reduced frame rate (subsample by factor 4) and stride likelihood context | VIPER · RLBench | task success | Improved |
| use VIPER rewards instead of task oracle sparse rewards | DreamerV3 · RLBench | task success | Improved |
| use VIPER rewards instead of task oracle dense rewards with state | DrQ / DreamerV3 · DMC | task return | Hurt |
| use VIPER instead of AMP adversarial reward (single-task and multi-task) | DrQ / DreamerV3 · DMC, Atari | task return | Improved |
Closest papers by meaning
- Goal-Driven Reward by Video Diffusion Models for Reinforcement Learning (CVPR 2026 · similarity 0.89)
- Diffusion Reward: Learning Rewards via Conditional Video Diffusion (ECCV 2024 · similarity 0.87)
- Reward Modeling for Computer-Using Agent from Video Execution (ECCV 2026 · similarity 0.87)
- DreamSmooth: Improving Model-based Reinforcement Learning via Reward Smoothing (ICLR 2024 · similarity 0.86)
- Improving Reward Model Generalization from Adversarial Process Enhanced Preferences (ICML 2025 · similarity 0.86)
- ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understanding (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
