Research map / Image generation / Text-to-image diffusion
VideoDPO: Omni-Preference Alignment for Video Diffusion Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts Direct Preference Optimization to text-to-video diffusion models using a multi-dimensional OmniScore to build best-vs-worst preference pairs, re-weighted by score-frequency, improving VideoCrafter2, T2V-Turbo, and CogVideo.
Problem
Text-to-video diffusion models often deviate from user preferences in visual quality and text-video semantic alignment, and existing preference alignment methods for video rely on single-aspect reward models or are costly to supervise.
Approach
Scores generated videos with a combined OmniScore (intra-frame quality, inter-frame consistency/motion, and text-video semantic alignment), forms a best-vs-worst preference pair per prompt automatically, and re-weights DPO loss by the inverse frequency of pair scores in a histogram so rarer, more distinctive pairs get more weight.
Key results
- On VideoCrafter2, VideoDPO raises VBench total from 80.44% to 81.93% and PickScore is unchanged at 20.65 versus baseline.
- On T2V-Turbo, VideoDPO improves VBench total from 80.95% to 81.80% and HPS (V) from 0.262 to 0.260 (lower), with PickScore rising from 21.15 to 21.18.
- Best-vs-worst pairing (one pair per prompt) outperforms better-vs-worse and best-vs-worse pairing strategies on VBench (81.93 vs 81.32 and 80.80).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply VideoDPO alignment | VideoCrafter2 · VBench | Total: 80.59 -> 81.93 | Helped |
| apply VideoDPO alignment | T2V-Turbo · VBench | Total: 80.95 -> 81.80 | Helped |
| apply VideoDPO alignment | CogVideo · VBench | Total: 79.30 -> 79.80 | Helped |
| disable preference-pair re-weighting (alpha 0.0) (ablation) | VideoCrafter2 VideoDPO · VBench | Total: 81.93 -> 80.89 | Hurt |
| set re-weighting alpha to 0.5 or 2.0 instead of 1.0 (ablation) | VideoCrafter2 VideoDPO · VBench | Total: 81.51; 81.93 vs. 81.93 | No clear effect |
| set re-weighting alpha to 2.0 instead of 1.0 (ablation) | VideoCrafter2 VideoDPO · VBench | Semantic alignment: 77.38 -> 79.59 | Helped |
| generate more videos per prompt (N=2, 3, or 4) (ablation) | VideoCrafter2 VideoDPO · VBench | Total: 80.89; 81.51; 81.93 | Helped |
| use Better-vs-Worse or Best-vs-Worse pairing instead of Best-vs-Worst (ablation) | VideoCrafter2 VideoDPO · VBench | Total: 81.32; 80.80 vs. 81.93 | Hurt |
| filter out the least-distinct preference pairs (ablation) | VideoCrafter2 VideoDPO · VBench | Total: 80.08; 81.29; 81.42 vs. 81.93 | Hurt |
| fine-tune only on winning samples rather than preference pairs (ablation) | VideoCrafter2 · VBench | Total: 78.78 vs. 81.93 | Hurt |
| use a single-aspect semantic or aesthetic reward rather than OmniScore (ablation) | VideoCrafter2 · VBench | Total: 80.20; 79.65 vs. 81.93 | Hurt |
| apply VideoDPO alignment | T2V-Turbo · HPS (V) | HPS (V): 0.262 -> 0.260 | Hurt |
Surprising findings
- Removing less-distinctive preference pairs (filtering) hurt performance, suggesting prompt diversity matters more than pair distinctiveness.
- On T2V-Turbo, HPS (V) slightly decreased after VideoDPO alignment (0.262 to 0.260) despite VBench gains.
Limitations
- Gains on human-preference metrics are small and mixed (e.g., CogVideo HPS (V) not reported; T2V-Turbo HPS slightly decreased).
- Number of generated videos N trades off performance and data-generation cost; optimal N is left open.
- Preference labels come from automatic scorers rather than human annotations.
Experimental setup
Datasets: VidProm
Benchmarks: VBench HPS (V) PickScore
Metrics: VBench total score VBench quality score VBench semantic score HPS v2 video score PickScore
Baselines: VideoCrafter-v2 T2V-Turbo CogVideoX VADER SFT
Builds on: Direct Preference Optimization Diffusion-DPO VBench VADER
Closest papers by meaning
- DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models (NeurIPS 2025 · similarity 0.92)
- DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Scheduling (CVPR 2025 · similarity 0.90)
- SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Dataset (NeurIPS 2024 · similarity 0.90)
- Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Search (NeurIPS 2025 · similarity 0.89)
- $\mathcal{V}ista\mathcal{DPO}$: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models (ICML 2025 · similarity 0.89)
- InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.