Research map / Image generation / Super-resolution
Diffusion Prior Interpolation for Flexibility Real-World Face Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Diffusion Prior Interpolation (DPI), a masked two-stage posterior sampling scheme with a learned condition Corrector (CRT) for real-world face super-resolution on a pre-trained DDPM.
Problem
Prior-based diffusion methods for face super-resolution produce realistic results but lack pixel-level consistency with the low-resolution input, while discriminative methods over-smooth; the paper seeks a flexible trade-off between consistency and diversity.
Approach
Mask the diffusion sampling with a fixed grid mask (strong constraint) in early steps and a time-weighted, edge-probability-based random mask (weak constraint) in later steps, and iteratively refine the intermediate condition with a learned Corrector network that is trained to predict ground-truth conditions.
Key results
- On CelebA1000 at x16, DPI reports FID 27.33 and LPIPS 0.1927 versus CodeFormer FID 34.15, with the best face-recognition accuracy of 72.8% at x16 on CelebA1000.
- On real-world datasets, DPI achieves the best IS on LFW-mild-type and WebPhoto (IS 3.67) and best MUSIQ/CLIPIQA on WIDER (64.90/0.7150) among the compared methods.
- Ablation at x8 shows removing CRT raises FID from 22.86 to 73.01, while removing refinement raises it only to 23.65.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CRT condition correction (ablation) | DPI · CelebA1000 test, ×8 | FID: 73.01 -> 22.86 | Helped |
| add CRT condition correction (ablation) | DPI · CelebA1000 test, ×8 | LPIPS: 0.2242 -> 0.1401 | Helped |
| add iterative condition refinement (ablation) | DPI · CelebA1000 test, ×8 | LPIPS: 0.1507 -> 0.1401 | No clear effect |
| add Stage II with weighted RACM (ablation) | DPI · CelebA1000 test, ×8 | FID: 53.60 -> 22.86 | Helped |
| add Stage II with weighted RACM (ablation) | DPI · CelebA1000 test, ×8 | SSIM: 0.7278 -> 0.6881 | Hurt |
| use the default weighted RACM rather than omitting RACM weight (ablation) | DPI · CelebA1000 test, ×8 | FID: 33.78 -> 22.86 | Helped |
| use k=2 rather than the varied sparsity setting (ablation) | DPI · CelebA1000 test, ×8 | FID: 28.09 -> 22.86 | Helped |
| use DPI instead of PGDiff | face super-resolution · WebPhoto medium | FID: 85.83 -> 81.77 | Helped |
| use DPI instead of PGDiff | face super-resolution · LFW mild | FID: 62.31 -> 65.91 | Hurt |
| use DPI instead of PGDiff | face super-resolution · WIDER heavy | FID: 47.63 -> 49.79 | Hurt |
Surprising findings
- Removing the CRT correction step degrades image quality far more than removing iterative condition refinement, which only marginally affects results.
- DPI is reported to be less effective on natural images (ImageNet) because RACM constraints weaken for complex backgrounds, so consistency is harder to ensure.
Limitations
- Relies on pre-trained face DDPM; less effective on natural images with complex backgrounds.
- Without supervised fine-tuning, consistency is hard to guarantee solely through conditional prior constraints (as stated in discussion).
Experimental setup
Datasets: FFHQ CelebA1000 FFHQ1000 LFW WebPhoto WIDER ImageNet
Benchmarks: CelebA1000 FFHQ1000 LFW WebPhoto WIDER
Metrics: LPIPS FID IS PSNR SSIM MUSIQ CLIPIQA face recognition accuracy (ACC) face recognition consistency distance (CS)
Baselines: CodeFormer DDNM DDRM SR3 ILVR DR2 DPS DiffFace PGDiff DFDNet GFPGAN VQFR
Builds on: DPS DDIM ILVR SR3 CodeFormer
Closest papers by meaning
- Taming Diffusion Prior for Image Super-Resolution with Domain Shift SDEs (NeurIPS 2024 · similarity 0.88)
- Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution (ICLR 2024 · similarity 0.87)
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.87)
- Simplified and Generalized Masked Diffusion for Discrete Data (NeurIPS 2024 · similarity 0.87)
- Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminator (NeurIPS 2025 · similarity 0.87)
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.