Research map / Image generation / Editing and style transfer
RetouchFormer: Semi-supervised High-Quality Face Retouching Transformer with Prior-Based Selective Self-Attention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RetouchFormer, a semi-supervised transformer for face retouching that predicts imperfection masks from a VQ feature dictionary and uses prior-based selective self-attention to synthesize normal skin in blemished regions.
Problem
Face retouching requires removing diverse, multi-scale facial imperfections while preserving identity and structure, but generic image enhancement methods fail to localize imperfections and typically need fully paired retouching data.
Approach
Treat face retouching as a soft inpainting task: a VQ dictionary of clean face priors yields an imperfection mask from the reconstruction difference, which gates a selective self-attention that queries imperfection tokens and attends only to normal-skin tokens; multi-scale patch tokens and unpaired data further help.
Key results
- On FFHQR (gamma=0.1), RetouchFormer reaches 45.13 dB PSNR and 0.0093 LPIPS, beating BPFRe by 0.56 dB PSNR.
- Versus RestoreFormer and SwinTransformer, RetouchFormer gains 2.66 and 2.84 dB PSNR at gamma=0.1.
- Removing the clean-face prior dictionary drops Soft-IoU from 0.3341 to 0.2833 (about 17.9% relative gain from the prior).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-scale tokens rather than using a single patch size (ablation) | RetouchFormer · FFHQR (γ=0.1) | PSNR: 40.20 -> 45.13 | Helped |
| use selective self-attention instead of global self-attention (ablation) | RetouchFormer · FFHQR (γ=0.1) | PSNR: 44.07 -> 45.13 | Helped |
| add unpaired training data Xu (ablation) | RetouchFormer · FFHQR (γ=0.1) | LPIPS: 0.0162 -> 0.0093 | Helped |
| add clean-face prior dictionary P to mask generation (ablation) | RetouchFormer · Soft-mask generation | Soft-IoU: 0.2833 -> 0.3341 | Helped |
Surprising findings
- Replacing selective self-attention with global self-attention drops PSNR by 1.06 dB, and single-scale patches drop it by 4.93 dB.
Limitations
- Quantitative evaluation requires ground-truth retouching pairs; FR-wild is evaluated only qualitatively.
- Paired retouching data is still needed for a portion of training (gamma fraction), with performance degrading as paired data shrinks.
Experimental setup
Datasets: FFHQR FR-wild Flickr-Faces-HQ
Benchmarks: FFHQR FR-wild
Metrics: PSNR SSIM LPIPS Soft-IoU
Baselines: Pix2PixHD MPRNet GPEN AutoRetouch ABPN BPFRe SwinTransformer RestoreFormer
Builds on: VQ-VAE RestoreFormer Pix2Pix Mean Teacher (EMA)
Closest papers by meaning
- RetouchGPT: LLM-based Interactive High-Fidelity Face Retouching via Imperfection Prompting (AAAI 2025 · similarity 0.91)
- VRetouchEr: Learning Cross-frame Feature Interdependence with Imperfection Flow for Face Retouching in Videos (CVPR 2024 · similarity 0.88)
- Self-Supervised Selective-Guided Diffusion Model for Old-Photo Face Restoration (NeurIPS 2025 · similarity 0.85)
- PFStorer: Personalized Face Restoration and Super-Resolution (CVPR 2024 · similarity 0.84)
- DiffRetouch: Using Diffusion to Retouch on the Shoulder of Experts (AAAI 2025 · similarity 0.84)
- FaceMe: Robust Blind Face Restoration with Personal Identification (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.