Research map / Video / Video segmentation and tracking
Dynamic Content Prediction with Motion-aware Priors for Blind Face Video Restoration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DCP-MP, a blind face video restoration framework that predicts discrete dictionary elements from degraded video using a motion-aware semantic dictionary (VQGAN-style) with motion regularization and a conditional skip-feature correction module.
Problem
Blind face video restoration: restoring high-quality face video sequences from unknown degradations while maintaining temporal consistency across frames, which image-based blind face restoration methods fail to handle.
Approach
A motion-aware semantic dictionary whose neighbor-set graph of element adjacencies across time encodes motion; a temporal-aware element predictor maps degraded content to dictionary elements, which are refined by cross-attention over motion neighbors, and a conditional structure feature correction module filters encoder skip features before the generator.
Key results
- On VFHQ-Test-LQ, DCP-MP reaches LPIPS 0.2418, FID 53.12, IFD 3.71, outperforming CodeFormer (LPIPS 0.3562, FID 65.68, IFD 9.86) and BasicVSR++ (IFD 5.11).
- On HDTF-LQ, DCP-MP achieves FID 26.24 and IFD 3.62 versus CodeFormer FID 34.14 and IFD 8.64.
- On YTF-Hard and YTF-Medium, DCP-MP has the lowest FID (54.10 and 52.81) and IFD (7.36 and 5.49) among compared methods; a 30-worker user study favored it on frame quality and temporal coherence.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DCP-MP instead of DifFace | Blind face video restoration · VFHQ-Test-LQ | FID: 63.73 -> 53.12 | Helped |
| Use DCP-MP instead of BasicVSR++ | Blind face video restoration · VFHQ-Test-LQ | IFD: 5.11 -> 3.71 | Helped |
| Use DCP-MP instead of DifFace | Blind face video restoration · HDTF-LQ | FID: 34.47 -> 26.24 | Helped |
| Use DCP-MP instead of PGTFormer | Blind face video restoration · HDTF-LQ | IFD: 4.55 -> 3.62 | Helped |
| Use DCP-MP instead of CodeFormer | Blind face video restoration · YTF-Medium | FID: 65.15 -> 52.81 | Helped |
| Use DCP-MP instead of BasicVSR++ | Blind face video restoration · YTF-Medium | IFD: 5.55 -> 5.49 | Helped |
| Use DCP-MP instead of CodeFormer | Blind face video restoration · YTF-Hard | FID: 72.25 -> 54.10 | Helped |
| Use DCP-MP instead of BasicVSR++ | Blind face video restoration · YTF-Hard | IFD: 7.40 -> 7.36 | Helped |
| Add the temporal-aware element predictor instead of nearest-neighbor dictionary matching (ablation) | DCP-MP · VFHQ-Test-LQ | LPIPS: 0.4512 -> 0.2418 | Helped |
| Add the temporal-aware element predictor instead of nearest-neighbor dictionary matching (ablation) | DCP-MP · HDTF-LQ | FID: 79.12 -> 26.24 | Helped |
| Add motion regularization (ablation) | DCP-MP · VFHQ-Test-LQ | IFD: 4.01 -> 3.71 | Helped |
| Add motion regularization (ablation) | DCP-MP · HDTF-LQ | IFD: 3.96 -> 3.62 | Helped |
Limitations
- Evaluation relies on synthetic degradations and a small in-the-wild set without ground truth
- Temporal consistency evaluated via IFD, a simple frame-difference metric
- Results on the full model show some metrics (e.g., IDS) not uniformly best in tables
Experimental setup
Datasets: VFHQ
Benchmarks: VFHQ-Test-LQ HDTF-LQ YTF-Medium YTF-Hard
Metrics: PSNR IDS LPIPS FID IFD
Baselines: BasicVSR++ MIA-VSR IA-RT FMA-Net PGTFormer CodeFormer RestoreFormer DR2 DiffBIR DifFace
Builds on: VQGAN CodeFormer BasicVSR++
Closest papers by meaning
- Discrete Prior-Based Temporal-Coherent Content Prediction for Blind Face Video Restoration (AAAI 2025 · similarity 0.93)
- Efficient Video Face Enhancement with Enhanced Spatial-Temporal Consistency (CVPR 2025 · similarity 0.86)
- Blind Face Restoration under Extreme Conditions: Leveraging 3D-2D Prior Fusion for Superior Structural and Texture Recovery (AAAI 2024 · similarity 0.84)
- $InterLCM$: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration (ICLR 2025 · similarity 0.84)
- ReF-LDM: A Latent Diffusion Model for Reference-based Face Image Restoration (NeurIPS 2024 · similarity 0.84)
- FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.