Research map / Image generation / Editing and style transfer
OSDFace: One-Step Diffusion Model for Face Restoration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OSDFace, a one-step diffusion model for blind face restoration that conditions Stable Diffusion on visual prompts from a VQ-based visual representation embedder, trained with identity, perceptual and GAN losses.
Problem
Multi-step diffusion face restoration is computationally expensive, and existing methods often produce disharmonious, unrealistic faces that lose identity consistency.
Approach
A visual representation embedder (VRE) tokenizes LQ faces with an LQ VQ dictionary to produce visual prompt embeddings replacing text prompts, combined with an ArcFace identity loss, edge-aware DISTS loss, and a GAN discriminator on diffusion latents, enabling single-step restoration.
Key results
- On synthetic CelebA-Test, OSDFace (one step) reaches LPIPS 0.3365 and DISTS 0.1773, the best DISTS among all compared methods, and the lowest Deg. (identity) score of 60.07.
- On real-world Wider-Test, OSDFace achieves FID 34.65 and MUSIQ 74.60, outperforming OSEDiff* retrained on FFHQ (FID 47.88).
- OSDFace runs in 0.10 s per 512x512 image with 2,132 GMACs, versus 9.03 s for DiffBIR (50 steps) and 0.13 s for OSEDiff, with 978.4M parameters vs 1,302M for OSEDiff.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add HQ-LQ feature association loss to VRE training (ablation) | OSDFace · Wider-Test | FID: 44.3360 -> 41.9502 | Helped |
| Replace learnable prompt embeddings with VRE (ablation) | OSDFace · Wider-Test | FID: 45.4041 -> 41.9502 | Helped |
| Replace DAPE prompt embeddings with VRE (ablation) | OSDFace · Wider-Test | FID: 45.9380 -> 41.9502 | Helped |
| Use EA-DISTS instead of EA-LPIPS (ablation) | OSDFace · Wider-Test | FID: 45.7834 -> 41.9502 | Helped |
| Use OSDFace instead of DiffBIR | face restoration · Wider-Test | FID: 35.343 -> 34.648 | Helped |
| Use OSDFace instead of DiffBIR | face restoration · LFW-Test | FID: 40.320 -> 44.629 | Hurt |
| Use OSDFace instead of PGDiff | face restoration · WebPhoto-Test | FID: 86.954 -> 84.597 | Helped |
Surprising findings
- Retraining OSEDiff on FFHQ (OSEDiff*) gives worse FID than the original OSEDiff on CelebA-Test (81.36 vs 73.48), suggesting the retraining hurts the one-step baseline.
Limitations
- Relies on Stable Diffusion 2.1-base and a VQ dictionary trained on FFHQ-style face data, so generalization beyond faces is only shown zero-shot on cartoons.
- Real-world evaluation relies on no-reference metrics and FID, which may not capture identity fidelity.
Experimental setup
Datasets: FFHQ FFHQ-retouched CelebA-Test Wider-Test LFW-Test WebPhoto-Test
Benchmarks: CelebA-Test Wider-Test LFW-Test WebPhoto-Test
Metrics: LPIPS DISTS MUSIQ NIQE FID Deg. LMD CLIPIQA MANIQA
Baselines: RestoreFormer++ VQFR CodeFormer DAEFR PGDiff DifFace DiffBIR OSEDiff
Builds on: Stable Diffusion OSEDiff D3SR DAEFR VQGAN
Closest papers by meaning
- FaceMe: Robust Blind Face Restoration with Personal Identification (AAAI 2025 · similarity 0.90)
- PFStorer: Personalized Face Restoration and Super-Resolution (CVPR 2024 · similarity 0.90)
- ReF-LDM: A Latent Diffusion Model for Reference-based Face Image Restoration (NeurIPS 2024 · similarity 0.88)
- Overcoming False Illusions in Real-World Face Restoration with Multi-Modal Guided Diffusion Model (ICLR 2025 · similarity 0.88)
- When StyleGAN Meets Stable Diffusion: a W+ Adapter for Personalized Image Generation (CVPR 2024 · similarity 0.88)
- Self-Supervised Selective-Guided Diffusion Model for Old-Photo Face Restoration (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.