Research map / Video / Video generation

T2V-Turbo-v2: Enhancing Video Model Post-Training through Data, Reward, and Conditional Guidance Design

ICLR 2025 · Video › Video generation · approach: Compositional text-to-video diffusion

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Distills a consistency model from VideoCrafter2 with reward feedback (HPSv2.1, CLIP, InternVideo2), curated training data, and motion guidance from MotionClone precomputed into training, reaching VBench Total 85.13 quality / 83.52 total.

Problem

Post-training diffusion text-to-video models to close the quality gap with proprietary systems, given that prior reward-based consistency distillation used only WebVid data and training-free guidance is costly at inference.

Approach

Extends T2V-Turbo by combining high-quality captioned data (VidGen-1M + WebVid), a diverse reward mix (HPSv2.1, CLIPScore, InternVideo2) applied only to short-caption data, and an energy-based motion guidance (MotionClone PTA) whose gradients are precomputed from training videos, with the target EMA network removed so full-model training fits in memory.

Key results

• The motion-guided variant reaches a VBench Total Score of 83.52 (Quality 85.13, Semantic 77.12) at 16 steps, above Gen-3 (82.32) and Kling (81.85).
• Motion guidance raises VBench Dynamic Degree from 61.39 to 90.00 while keeping Motion Smoothness near 97.
• Training the CD loss on the full dataset while restricting reward optimization to short-caption WebVid data yields the best Total Score (82.93 w/o MG vs. 81.34 for WebVid-only training with the same method), suggesting dense captions limit reward gains.

Measured effects reported in the paper

Surprising findings

• Reward optimization on the high-quality, long-caption datasets (OpenVid, VidGen) gave only modest gains, while the same reward objective on WebVid gave large gains for the VCM baseline.
• Applying reward optimization on the high-quality VidGen data alone caused reward over-optimization and color distortion, so the CD loss must act as a regularizer.

Limitations

• Reward models (CLIP, InternV2 with ~40 tokens, HPSv2.1 with 77 tokens) have short context limits, so high-quality dense-caption datasets like OpenVid-1M cannot be fully exploited.
• The teacher VideoCrafter2 uses a CLIP text encoder, which limits handling of dense captions.
• Motion guidance requires precomputing DDIM inversion over the training set, and the LAS guidance term is not distilled.

Experimental setup

Datasets: VidGen-1M WebVid-10M OpenVid-1M

Benchmarks: VBench T2V-CompBench

Metrics: VBench Total Score Quality Score Semantic Score Dynamic Degree Motion Smoothness

Baselines: T2V-Turbo VideoCrafter2 Gen-3 Kling Gen-2 Pika VideoLCM

Builds on: T2V-Turbo VideoLCM MotionClone consistency models classifier-free guidance

Closest papers by meaning

• T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback (NeurIPS 2024 · similarity 0.92)
• Encapsulated Composition of Text-to-Image and Text-to-Video Models for High-Quality Video Synthesis (CVPR 2025 · similarity 0.88)
• PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generation (CVPR 2025 · similarity 0.87)
• VideoGuide: Improving Video Diffusion Models without Training Through a Teacher's Guide (CVPR 2025 · similarity 0.87)
• Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Control (NeurIPS 2026 · similarity 0.87)
• Enhancing Motion in Text-to-Video Generation with Decomposed Encoding and Conditioning (NeurIPS 2024 · similarity 0.87)