Research map / Video / Video generation
DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DenseDPO, which builds video DPO pairs by denoising partially noised copies of ground-truth videos and labels preferences per 1s segment, reducing motion bias and enabling GPT-o3 segment-level automatic labeling.
Problem
Vanilla DPO for text-to-video diffusion uses pairs from independent noise, and clip-level human preferences favor low-motion videos with fewer artifacts, suppressing dynamic motion; VLMs also struggle to judge long videos.
Approach
Generate structurally aligned video pairs by denoising guided corruptions of a ground-truth video (SDEdit-style), then collect preference labels per 1-second temporal segment instead of per whole clip, giving dense supervision in a DPO objective and allowing short-segment VLM labeling.
Key results
- DenseDPO with 10k human segment labels matches vanilla DPO (30k labels) on text alignment, visual quality and temporal consistency while significantly raising VideoJAM-bench dynamic degree (85.38 vs 80.25 VBench).
- GPT-o3 segment-level labels reach 70.59% accuracy on 5s long-video preference vs 53.45% for GPT-o3 on whole videos; off-the-shelf VLM-labeled DenseDPO matches human-label DenseDPO on several metrics.
- StructuralDPO alone preserves dynamic degree but reduces visual quality and text alignment relative to vanilla DPO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace independent-noise whole-video comparisons with aligned video pairs and segment-level preference labels | Vanilla DPO · VideoJAM-bench | Dynamic degree (VBench): 80.25 -> 85.38 | Helped |
| replace independent-noise whole-video comparisons with aligned video pairs and segment-level preference labels | Vanilla DPO · MotionBench | Dynamic degree (VBench): 72.55 -> 84.73 | Helped |
| use aligned pairs with segment-level preferences instead of structurally similar pairs with binary preferences | Structural DPO · VideoJAM-bench | Visual quality: 0.341 -> 0.376 | Helped |
| use aligned pairs with segment-level preferences instead of structurally similar pairs with binary preferences | Structural DPO · MotionBench | Dynamic degree (VBench): 79.95 -> 84.73 | Helped |
| aggregate segment labels into a single binary preference by majority voting (ablation) | StructuralDPO · VideoJAM-bench | Text alignment: 0.843 -> 0.847 | No clear effect |
| randomly flip 20% or 40% of preference labels (ablation) | DenseDPO · VideoJAM-bench | Text alignment: 0.863 -> 0.846; 0.863 -> 0.782 | Hurt |
| use 0.5× segment preference labels (ablation) | DenseDPO · VideoJAM-bench | Text alignment: 0.863 -> 0.806 | Hurt |
| increase segment preference labels to 2× (ablation) | DenseDPO · VideoJAM-bench | Text alignment: 0.863 -> 0.882 | Helped |
| use a different, non-overlapping set of 10k high-quality guidance videos (ablation) | DenseDPO · VideoJAM-bench | Text alignment: 0.863 -> 0.868 | No clear effect |
| shorten preference segments from 1s to 0.5s (ablation) | DenseDPO · VideoJAM-bench | Text alignment: 0.806 -> 0.811 | No clear effect |
| lengthen preference segments from 1s to 2s (ablation) | DenseDPO · VideoJAM-bench | Text alignment: 0.806 -> 0.795 | Hurt |
| increase GPT o3 segment-level labels from 10k to 55k (ablation) | DenseDPO · VideoJAM-bench | Text alignment: 0.782 -> 0.842 | Helped |
Surprising findings
- Structural (guided-pair) DPO can cause the model to unlearn the real data distribution, attributed to the gradient from losing samples dominating on shared ground-truth pixels.
- Fine-tuned video reward models perform better than GPT-o3 on 1s segments but degrade sharply on 5s long videos, where GPT-o3 segment aggregation performs best.
Limitations
- Over 60% of pairs have both winning and losing segments, and ties handling is heuristic
- Guided sampling reduces diversity and can cause the model to diverge from the real data distribution
- VLM labels are noisy; DenseDPO with 10k GPT labels yields only marginal gains over the pre-trained model
Experimental setup
Datasets: VideoJAM-bench MovieGenBench curated ~55k text-video pairs from existing large-scale video datasets
Benchmarks: VideoJAM-bench MotionBench
Metrics: VBench aesthetic quality imaging quality subject consistency background consistency motion smoothness dynamic degree VisionReward text alignment visual quality temporal consistency preference accuracy
Baselines: Vanilla DPO SFT Pre-trained model D3PO DPOK REFL SePPO VisionReward VideoScore LiFT VideoReward
Builds on: Diffusion-DPO D3PO rectified flow SDEdit VideoAlign
Closest papers by meaning
- VideoDPO: Omni-Preference Alignment for Video Diffusion Generation (CVPR 2025 · similarity 0.92)
- $\mathcal{V}ista\mathcal{DPO}$: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models (ICML 2025 · similarity 0.89)
- Calibrated Multi-Preference Optimization for Aligning Diffusion Models (CVPR 2025 · similarity 0.88)
- Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Search (NeurIPS 2025 · similarity 0.88)
- Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimization (NeurIPS 2025 · similarity 0.88)
- Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.