Research map / Video / Video generation
Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Upscale-A-Video adapts the SD x4 Upscaler into a text-guided latent video diffusion model for real-world video super-resolution, combining temporal layers in U-Net and VAE-decoder with a training-free flow-guided recurrent latent propagation module.
Problem
Real-world video super-resolution with diffusion priors suffers from temporal inconsistency and flickering caused by stochastic diffusion sampling and the VAE decoder, especially over long videos.
Approach
Inflate a pretrained image x4 upscaler's U-Net and VAE-decoder with temporal layers (local consistency), then add a training-free bidirectional flow-guided latent propagation during denoising (global consistency), plus text-prompt and noise-level controls for fidelity-quality trade-off.
Key results
- Achieves highest PSNR on all four synthetic datasets (e.g., 30.79 on UDM10 vs 29.60 for DBVSR).
- On VideoLQ, CLIP-IQA 0.530 and DOVER 7.811 are the best reported, above RealBasicVSR's 0.387 and 7.562.
- Ablation on YouHQ40: finetuned VAE-decoder plus latent propagation reduces flow warping error from 2.398 to 0.737 and raises PSNR from 23.82 to 25.83.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Upscale-A-Video instead of DBVSR | Video super-resolution · SPMCS | PSNR: 24.28 -> 25.32 | Helped |
| Use Upscale-A-Video instead of RealBasicVSR | Video super-resolution · SPMCS | LPIPS: 0.198 -> 0.222 | Hurt |
| Use Upscale-A-Video instead of DBVSR | Video super-resolution · UDM10 | PSNR: 29.60 -> 30.79 | Helped |
| Use Upscale-A-Video instead of DBVSR | Video super-resolution · UDM10 | LPIPS: 0.155 -> 0.133 | Helped |
| Use Upscale-A-Video instead of DBVSR | Video super-resolution · REDS30 | PSNR: 24.37 -> 24.41 | Helped |
| Use Upscale-A-Video instead of RealBasicVSR | Video super-resolution · REDS30 | LPIPS: 0.249 -> 0.335 | Hurt |
| Use Upscale-A-Video instead of DBVSR | Video super-resolution · YouHQ40 | PSNR: 25.37 -> 25.83 | Helped |
| Use Upscale-A-Video instead of StableSR | Video super-resolution · YouHQ40 | LPIPS: 0.271 -> 0.268 | Helped |
| Use Upscale-A-Video instead of RealBasicVSR | Video super-resolution · VideoLQ | CLIP-IQA: 0.387 -> 0.530 | Helped |
| Use Upscale-A-Video instead of RealBasicVSR | Video super-resolution · AIGC30 | DOVER: 10.68 -> 11.67 | Helped |
| Add the finetuned VAE-Decoder (ablation) | Upscale-A-Video · YouHQ40 | PSNR: 23.82 -> 25.47 | Helped |
| Add the finetuned VAE-Decoder (ablation) | Upscale-A-Video · YouHQ40 | Ewarp ×10−3: 1.815 -> 0.842 | Helped |
Surprising findings
- The SD x4 Upscaler baseline performs poorly on real-world VSR because it was not trained on mixed degradations.
- Flow warping error of the proposed method on REDS30 is not best; RealBasicVSR and DBVSR are competitive or better on some temporal metrics.
Limitations
- Latent propagation must be applied with user-chosen steps and is tuned per video type (early for minor jitter, late for AIGC videos).
- Relies on optical flow validity via forward-backward consistency, which may fail under occlusion or large motion.
- Diffusion sampling remains expensive and the paper gives no inference speed analysis.
Experimental setup
Datasets: WebVid10M YouHQ SPMCS UDM10 REDS30 VideoLQ AIGC30
Benchmarks: SPMCS UDM10 REDS30 YouHQ40 VideoLQ AIGC30
Metrics: PSNR SSIM LPIPS flow warping error CLIP-IQA MUSIQ DOVER
Baselines: Real-ESRGAN SD x4 Upscaler ResShift StableSR RealVSR DBVSR RealBasicVSR
Builds on: Stable Diffusion x4 Upscaler RAFT StableSR RealBasicVSR classifier-free guidance
Closest papers by meaning
- One-Step Diffusion for Detail-Rich and Temporally Consistent Video Super-Resolution (NeurIPS 2025 · similarity 0.90)
- Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Search (NeurIPS 2025 · similarity 0.89)
- SeeClear: Semantic Distillation Enhances Pixel Condensation for Video Super-Resolution (NeurIPS 2024 · similarity 0.89)
- VideoElevator: Elevating Video Generation Quality with Versatile Text-to-Image Diffusion Models (AAAI 2025 · similarity 0.89)
- VideoGuide: Improving Video Diffusion Models without Training Through a Teacher's Guide (CVPR 2025 · similarity 0.89)
- MeDM: Mediating Image Diffusion Models for Video-to-Video Translation with Temporal Correspondence Guidance (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.