Research map / Image generation / Image restoration
Learning Degradation-unaware Representation with Prior-based Latent Transformations for Blind Face Restoration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PLTrans, a transformer-based blind face restoration method that learns a degradation-unaware query via latent diffusion regularization and prior-based key/value from a VQ latent dictionary, trained on FFHQ.
Problem
Blind face restoration must recover high-fidelity face details and preserve identity from images with unknown, complex degradations, and existing methods generalize poorly to real-world degradation.
Approach
Decouple the transformer attention: a latent diffusion module with DWT low-frequency injection produces a degradation-insensitive query, while a VQ latent dictionary refined by a learnable global mapping over top-d nearest elements produces key and value for cross-attention.
Key results
- On CelebA-Test at x64 downsampling, PLTrans achieves FID 39.58 and PSNR 19.36, versus the second-best GPEN (FID 45.12) and DIL (PSNR 19.25).
- Ablation on CelebA-Test: removing the latent diffusion module raises FID from 39.58 to 44.96, and removing the DWT injection lowers IDS from 0.5264 to 0.4672.
- A user study (80 workers, 50 images from in-the-wild sets) gives PLTrans the highest average score among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PLTrans instead of GPEN | Blind face restoration · CelebA-Test ×64 | FID: 45.12 -> 39.58 | Helped |
| Use PLTrans instead of DIL | Blind face restoration · CelebA-Test ×64 | PSNR: 19.25 -> 19.36 | Helped |
| Add latent dictionary V (ablation) | PLTrans · CelebA-Test | LPIPS: 0.4808 -> 0.4260 | Helped |
| Add latent diffusion-based regularization (ablation) | PLTrans · CelebA-Test | FID: 44.96 -> 39.58 | Helped |
| Add Lightweight Transformer to the prior-based key-value module (ablation) | PLTrans · CelebA-Test | PSNR: 19.11 -> 19.36 | Helped |
| Add DWT-based low-frequency information injection (ablation) | PLTrans · CelebA-Test | IDS: 0.4672 -> 0.5264 | Helped |
Surprising findings
- Enhancing key and value yields only a slight quality gain, while improving the query has a larger effect, motivating the query-focused design.
Limitations
- Generalization is evaluated mainly on synthetic degradations plus in-the-wild sets scored by a user study rather than standard no-reference metrics.
- Latent dictionary may not generalize well to degraded inputs, as the authors note.
Experimental setup
Datasets: FFHQ CelebA-HQ WIDER FACE LFW WebPhoto-Test
Benchmarks: CelebA-Test WIDER-Hard WIDER-Medium LFW-Test WebPhoto-Test
Metrics: FID LPIPS PSNR IDS user score
Baselines: GFP-GAN GPEN RestoreFormer Panini Restormer DIL ILVR DR2
Builds on: VQGAN ILVR DDPM RestoreFormer
Closest papers by meaning
- ReF-LDM: A Latent Diffusion Model for Reference-based Face Image Restoration (NeurIPS 2024 · similarity 0.88)
- $InterLCM$: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restoration (ICLR 2025 · similarity 0.87)
- Blind Face Restoration under Extreme Conditions: Leveraging 3D-2D Prior Fusion for Superior Structural and Texture Recovery (AAAI 2024 · similarity 0.87)
- OSDFace: One-Step Diffusion Model for Face Restoration (CVPR 2025 · similarity 0.87)
- FaceMe: Robust Blind Face Restoration with Personal Identification (AAAI 2025 · similarity 0.86)
- Learning from History: Task-agnostic Model Contrastive Learning for Image Restoration (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.