Research map / Video / Video generation
Rethinking Reward Signals in Video GRPO: When Scores Become Targets
Rui Li, Yuanzhi Liang, Ziqi Ni, Haibin Huang, Chi Zhang, Xuelong Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace DanceGRPO summed reward with TaRoS reward signaling (Qwen2.5VL-72B) | Wan2.1-T2V-1.3B · VBench | Total score: 79.72 -> 80.56 | Improved |
| replace DanceGRPO with TaRoS (VideoAlign reward) | Wan2.1-T2V-1.3B · VBench | Total score: 79.46 -> 79.84 | Improved |
| scale TaRoS reward model from Qwen2.5VL-7B to 72B | TaRoS on Wan2.1-T2V-1.3B · VBench | Total score: 79.90 -> 80.56 | Improved |
| swap DanceGRPO reward model from VideoAlign to Qwen2.5VL-72B | DanceGRPO on Wan2.1-T2V-1.3B · VBench | Total score: 79.46 -> 79.72 | No clear effect |
| apply TaRoS with Qwen2.5VL-72B to Wan2.1-14B | Wan2.1-T2V-14B · VBench | Total score: 81.46 -> 82.09 | Improved |
| apply TaRoS with VideoAlign to Wan2.1-14B | Wan2.1-T2V-14B · VBench | Total score: 81.46 -> 81.58 | No clear effect |
| apply TaRoS with VideoAlign to Wan2.1-14B | Wan2.1-T2V-14B · VBench | Quality score: 84.03 -> 83.61 | Hurt |
| apply TaRoS (Qwen2.5VL-72B) to HunyuanVideo | HunyuanVideo · VBench | Image quality: 64.61 -> 68.83 | Improved |
| apply TaRoS (Qwen2.5VL-72B) to HunyuanVideo | HunyuanVideo · VBench | Spatial relationship: 67.55 -> 69.84 | Improved |
| add third reward dimension to DanceGRPO (ablation) | DanceGRPO · Reward ablation, iStock 1.4k prompts | VideoAlign VQ: 3.493 -> 3.412 | Hurt |
| add third reward dimension to TaRoS (ablation) | TaRoS · Reward ablation, iStock 1.4k prompts | VideoAlign TA: -1.312 -> -0.7114 | Improved |
| use TaRoS instead of DanceGRPO with three reward dimensions (ablation) | Wan2.1-T2V-1.3B · Reward ablation, iStock 1.4k prompts | VideoAlign VQ: 3.412 -> 3.501 | Improved |
Closest papers by meaning
- TAGRPO: Boosting GRPO on Image-to-Video Generation with Direct Trajectory Alignment (ICML 2026 · similarity 0.89)
- DiverseGRPO: Mitigating Mode Collapse in Image Generation via Diversity-Aware GRPO (CVPR 2026 · similarity 0.88)
- Diverse Video Generation with Determinantal Point Process-Guided Policy Optimization (CVPR 2026 · similarity 0.88)
- Improving Video Generation with Human Feedback (NeurIPS 2025 · similarity 0.87)
- Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation (NeurIPS 2025 · similarity 0.86)
- DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
