Research map / Image generation / Image restoration

Zero-shot Video Restoration and Enhancement Using Pre-Trained Image Diffusion Model

AAAI 2025 · Image generation › Image restoration · approach: Diffusion-based universal restoration · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ZVRD, a training-free plug-and-play framework that adapts pre-trained zero-shot image diffusion restoration methods (DDNM, GDP, DiffBIR) to video via SLR temporal attention, temporal consistency guidance, spatial-temporal noise sharing, and early stopping.

Problem

Zero-shot diffusion-based image restoration and enhancement methods produce severe temporal flickering when applied frame-by-frame to degraded videos, and existing video restoration methods need task-specific training.

Approach

Replace spatial self-attention in the U-Net decoder with SLR temporal attention (cross-neighbor-frame attention plus self-corrected trajectory attention using flow, similarity, and historically-best trajectories), and add pixel- and semantic-level consistency gradient guidance, noise sharing across frames, and early stopping of sampling, all without training.

Key results

• On 4x video super-resolution, GDP+ZVRD reduces warping error from 4.0327% to 0.4234% and improves PSNR from 20.44 to 21.39 dB over GDP.
• DDNM+ZVRD reaches FID 106.84 versus 110.13 for DDNM and outperforms supervised VRT on FID, with WE reduced to about one third of DDNM's.
• On low-light video enhancement, GDP+ZVRD attains the best PSNR (17.56), SSIM (0.8237) and WE (0.3352%) among compared methods.

Measured effects reported in the paper

Surprising findings

• Adding ZVRD to DDNM slightly lowers PSNR/SSIM on some tasks relative to baseline in the tables, showing temporal gains come with trade-offs in fidelity metrics.
• GDP+ZVRD has a higher OFME (0.9948) than GDP+SLTA-free variants in the ablation trajectory, and OFME is not uniformly improved across backbones (e.g., DDNM+ZVRD OFME 0.2596 vs VRT 0.1563).

Limitations

• Frames are center-cropped and resized to 256x256 due to slow DDPM sampling, so arbitrary-size videos are not directly handled.
• Pixel-level consistency and self-corrected attention rely on optical flow computed from intermediate x0 predictions, which can be inaccurate for heavily degraded inputs.
• Self-corrected trajectory attention is applied only to the 32x32 resolution layers; higher-resolution features are handled only by guidance.

Experimental setup

Datasets: REDS4 Vid4 UDM10 REDS Set8 DAVIS Videvo20 DID

Benchmarks: REDS4 Vid4 UDM10 REDS Set8 DAVIS Videvo20 DID

Metrics: PSNR SSIM FID warping error frame similarity optical flow map error

Baselines: VRT DDNM GDP DPS BiSTNet FastLLVE SGZ DiffBIR DiffIR2VR RePaint FastDVDNet UDVD

Builds on: DDNM GDP DPS RAFT

Closest papers by meaning

• Reconciling Stochastic and Deterministic Strategies for Zero-shot Image Restoration using Diffusion Model in Dual (CVPR 2025 · similarity 0.91)
• TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models (CVPR 2024 · similarity 0.87)
• Zero-Shot Image Restoration Using Few-Step Guidance of Consistency Models (and Beyond) (CVPR 2025 · similarity 0.86)
• VideoGuide: Improving Video Diffusion Models without Training Through a Teacher's Guide (CVPR 2025 · similarity 0.86)
• Motion-Zero: A Zero-Shot Trajectory Control Framework of Moving Object for Diffusion-Based Video Generation (AAAI 2025 · similarity 0.86)
• Learning Diffusion Texture Priors for Image Restoration (CVPR 2024 · similarity 0.86)