Research map / Image generation / Super-resolution
DOVE: Efficient One-Step Diffusion Model for Real-World Video Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes CogVideoX1.5 into DOVE, a one-step diffusion model for real-world video super-resolution, using a two-stage latent-then-pixel regression training strategy and a new curated dataset HQ-VSR.
Problem
Multi-step diffusion-based video super-resolution is extremely slow, and one-step VSR is hard due to the high cost of video training for distillation methods and the instability of adversarial training under strict fidelity demands.
Approach
Start one-step denoising from the bilinearly upscaled LR latent at a small timestep (t=399) of a pretrained T2V model, train with regression losses only (latent MSE, then pixel MSE + DISTS + frame-difference loss with mixed image/video data), and curate a VSR-specific dataset via a filtering and motion-cropping pipeline.
Key results
- On UDM10, DOVE reaches PSNR 26.48 and LPIPS 0.2696, versus 24.23 PSNR and 0.3272 LPIPS for the next-best MGLD-VSR.
- DOVE is about 28x faster than MGLD-VSR in running time on an A100 for a 33-frame 720x1280 video, and about 8x faster than VEnhancer.
- Stage-2 mixed image/video training improves LPIPS from 0.3037 (stage-1 only) to 0.2696 and DOVER from 0.6154 to 0.7809 on UDM10.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add pixel-space stage-2 after latent-space stage-1 (ablation) | DOVE · UDM10 | LPIPS: 0.3037 -> 0.2784 | Helped |
| add pixel-space stage-2 after latent-space stage-1 (ablation) | DOVE · UDM10 | PSNR: 27.20 -> 26.39 | Hurt |
| add video samples to image-only stage-2 training (ablation) | DOVE · UDM10 | DOVER: 0.7694 -> 0.7809 | Helped |
| set stage-2 image ratio to 80% rather than video-only (ablation) | DOVE · UDM10 | DOVER: 0.7647 -> 0.7809 | Helped |
| set stage-2 image ratio to 80% rather than image-only (ablation) | DOVE · UDM10 | PSNR: 26.39 -> 26.48 | Helped |
| train stage-1 on HQ-VSR instead of YouHQ (ablation) | DOVE · UDM10 | PSNR: 26.88 -> 27.20 | Helped |
| train stage-1 on HQ-VSR instead of OpenVid-1M (ablation) | DOVE · UDM10 | LPIPS: 0.3376 -> 0.3037 | Helped |
| add metadata, scene, and quality filtering to OpenVid-1M (ablation) | DOVE training data · UDM10 | LPIPS: 0.3376 -> 0.3236 | Helped |
| add motion processing to filtered OpenVid-1M (ablation) | DOVE training data · UDM10 | DOVER: 0.5357 -> 0.6154 | Helped |
| use one-step DOVE instead of MGLD-VSR | real-world video super-resolution · 33-frame 720×1280 video on A100 | inference speed: 28× | Helped |
Surprising findings
- Stage-1 latent-only MSE training gives the highest PSNR (27.20) on UDM10 but much worse perceptual scores, so pixel-space refinement trades a small PSNR drop for large perceptual gains.
Limitations
- Ablations are evaluated on UDM10 only, and the stage-2 gains are modest in PSNR.
- Data for the video pipeline depends on OpenVid-1M and multiple external quality metrics, which may limit reproducibility.
Experimental setup
Datasets: HQ-VSR DIV2K OpenVid-1M UDM10 SPMCS YouHQ40 RealVSR MVSR4x VideoLQ
Benchmarks: UDM10 SPMCS YouHQ40 RealVSR MVSR4x VideoLQ
Metrics: PSNR SSIM LPIPS DISTS CLIP-IQA FasterVQA DOVER E*warp
Baselines: RealESRGAN ResShift RealBasicVSR Upscale-A-Video MGLD-VSR VEnhancer STAR
Builds on: CogVideoX RealBasicVSR degradation pipeline Real-ESRGAN degradation DISTS
Closest papers by meaning
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.91)
- SkipSR: Faster Super-Resolution with Token Skipping (NeurIPS 2026 · similarity 0.90)
- One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution (NeurIPS 2025 · similarity 0.89)
- Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs (NeurIPS 2024 · similarity 0.89)
- One Diffusion Step to Real-World Super-Resolution via Flow Trajectory Distillation (ICML 2025 · similarity 0.89)
- TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.