Research map / Image generation / Editing and style transfer
Diffusion face generation and editing
Generate and edit identity-consistent faces using diffusion and 3D priors.
61 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Editing and style transfer, keyword cluster “face · facial · identity”
State History Tried Baselines Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
26 papers in 2023–24 and 29 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 9.5% to 7.4%.
Open questions the data points to
- Papers report FaceShifter on FF++ (Pose) anywhere from 1.62 to 2.19: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Synthesizing Normalized Faces From Facial Identity Features (CVPR 2017 · 149 citations)
- 2018 · Face Sketch Synthesis From Coarse to Fine (AAAI 2018 · 18 citations)
- 2019 · Dual Variational Generation for Low Shot Heterogeneous Face Recognition (NeurIPS 2019 · 8 citations)
- 2020 · Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning (CVPR 2020 · 336 citations)
- 2021 · Pseudo Facial Generation With Extreme Poses for Face Recognition (CVPR 2021 · 7 citations)
- 2023 · DCFace: Synthetic Face Generation With Dual Condition Diffusion Model (CVPR 2023 · 150 citations)
- 2024 · IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representation (CVPR 2024 · 25 citations)
- 2025 · StableAnimator: High-Quality Identity-Preserving Human Image Animation (CVPR 2025 · 25 citations)
- 2026 · Non-Colliding Biometric Identities for Digital Entities: Geometry, Capacity, and Million-Scale Virtual Identity Provisioning (NeurIPS 2026 · 0 citations)
Earliest papers here
- Synthesizing Normalized Faces From Facial Identity Features (CVPR 2017 · 149 citations)
- Face Sketch Synthesis From Coarse to Fine (AAAI 2018 · 18 citations)
- Dual Variational Generation for Low Shot Heterogeneous Face Recognition (NeurIPS 2019 · 8 citations)
What has been tried
51 papers here report 533 measured changes (a component added, removed or swapped, compared with a baseline): helped 73%, no clear effect 5%, hurt 20%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add face masks to the loss | Unseen100 · CSIM | 0.639 -> 0.805 | Helped | StableAnimator: High-Quality Identity-Preserving Human Image Animation (CVPR 2025 · 25 citations) |
| Fine-tune the CLIP encoder | Pixabay test set · DINO score | 90.385 -> 92.695 | Helped | IMPRINT: Generative Object Compositing by Learning Identity-Preserving Representation (CVPR 2024 · 25 citations) |
| Add explicit background conditioning | CelebA-HQ · ID. (×100) | 81.5 -> 87.0 | Helped | Towards a Simultaneous and Granular Identity-Expression Control in Personalized Face Generation (CVPR 2024 · 23 citations) |
| Use specialized age-estimation loss functions instead of cross-entropy after pretraining | Intra-dataset age estimation · MAE | No clear effect | A Call to Reflect on Evaluation Practices for Age Estimation: Comparative Analysis of the State-of-the-Art and a Unified Benchmark (CVPR 2024 · 22 citations) | |
| add global and local identity features | HumanIPHC 10 identities · ID sim. | 0.3915 -> 0.8429 | Helped | CapHuman: Capture Your Moments in Parallel Universes (CVPR 2024 · 19 citations) |
| Add style modulation network T to mapping network M | CelebAMask-HQ, text + semantic mask · FID | 62.08 -> 44.91 | Helped | Diffusion-driven GAN Inversion for Multi-Modal Face Image Generation (CVPR 2024 · 18 citations) |
| use the MSID encoder instead of ArcFace multi-scale features | FaceForensics++ · CLIP | 0.2069 to 0.2486 | Helped | Face2Diffusion for Fast and Editable Face Personalization (CVPR 2024 · 18 citations) |
| personalize the restoration model using reference images and adapter blocks with learnable γ | Celeb-Ref heavy degradation · ID | 33.90 -> 57.18 | Helped | PFStorer: Personalized Face Restoration and Super-Resolution (CVPR 2024 · 17 citations) |
| Add the refinement module to Re-PoI | CelebA-HQ test · MSE | 0.0404 -> 0.0252 | Helped | Controllable 3D Face Generation with Conditional Style Code Diffusion (AAAI 2024 · 16 citations) |
| Add geometry-centric SDS using texture-less rendering | 3D face geometry · geometric detail | Helped | Text-Guided 3D Face Synthesis - From Generation to Editing (CVPR 2024 · 15 citations) | |
| use incomplete UV texture instead of the original image as the condition | FFHQ · LPIPS | 0.1625 -> 0.1575 | Helped | UV-IDM: Identity-Conditioned Latent Diffusion Model for Face UV-Texture Generation (CVPR 2024 · 14 citations) |
| Add HQ-LQ feature association loss to VRE training | Wider-Test · FID | 44.3360 -> 41.9502 | Helped | OSDFace: One-Step Diffusion Model for Face Restoration (CVPR 2025 · 13 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| FaceShifter | FF++ | Pose | 1.62 – 2.19 | 2 papers, e.g. CVPR 2024 and CVPR 2023 |
What reviewers will expect
Datasets and baselines named in the 21 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- FFHQ · 7 of 21 papers
- CelebA-HQ · 4 of 21 papers
- WebPhoto-Test · 3 of 21 papers
- LFW · 3 of 21 papers
- LFW-Test · 2 of 21 papers
- WIDER-Test · 2 of 21 papers
- CASIA-WebFace · 2 of 21 papers
- CFP-FP · 2 of 21 papers
Baselines
- InstantID · 5 of 21 papers
- IP-Adapter · 4 of 21 papers
- PhotoMaker · 4 of 21 papers
- CodeFormer · 3 of 21 papers
- VQFR · 3 of 21 papers
- DCFace · 3 of 21 papers
- GPEN · 2 of 21 papers
- DifFace · 2 of 21 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 55 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 4 papers
- Alibaba Group · 3 papers
- Zhejiang University · 3 papers
- Fudan University · 3 papers
- Nankai University · 3 papers
- Tencent · 3 papers
- Shanghai Collaborative Innovation Center of Intelligent Visual Computing · 2 papers
- Microsoft Research Asia · 2 papers
- Samsung · 2 papers
- Michigan State University · 2 papers
Use it
Download BibTeX of 28 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Identity-preserving face manipulation (88 papers)
- Talking face generation (21 papers)
Elsewhere in the topic
- Instruction-based diffusion editing (151 papers)
- Texture and sketch synthesis (130 papers)
- Image-to-image translation (127 papers)
- Arbitrary neural style transfer (124 papers)
- Inversion-based diffusion editing (113 papers)
- Localized and attribute-specific editing (73 papers)
- Virtual try-on (46 papers)
- Pose-guided person image synthesis (29 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).