Research map / 3D vision / Face and shape
Face reflectance and relightable rendering
Recovers facial reflectance and lighting with inverse and neural rendering.
85 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Face and shape, keyword cluster “3d face · face reconstruction · faces”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Few recent reports The tracked benchmarks have at most one report from 2025–26 that could beat the old record, too few to say whether they are still moving.
25 papers in 2023–24 and 9 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 6.9% to 2.5%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Adaptive 3D Face Reconstruction From Unconstrained Photo Collections (CVPR 2016 · 152 citations)
- 2017 · Learning Detailed Face Reconstruction From a Single Image (CVPR 2017 · 376 citations)
- 2018 · SfSNet: Learning Shape, Reflectance and Illuminance of Faces `in the Wild' (CVPR 2018 · 301 citations)
- 2019 · GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction (CVPR 2019 · 332 citations)
- 2020 · RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild (CVPR 2020 · 1,623 citations)
- 2021 · Face Forgery Detection by 3D Decomposition (CVPR 2021 · 106 citations)
- 2022 · RigNeRF: Fully Controllable Neural 3D Portraits (CVPR 2022 · 129 citations)
- 2023 · DiffusionRig: Learning Personalized Priors for Facial Appearance Editing (CVPR 2023 · 67 citations)
- 2024 · 3D Facial Expressions through Analysis-by-Neural-Synthesis (CVPR 2024 · 48 citations)
- 2025 · Do Not DeepFake Me: Privacy-Preserving Neural 3D Head Reconstruction Without Sensitive Images (AAAI 2025 · 4 citations)
- 2026 · WildCap: Facial Albedo Capture in the Wild via Hybrid Inverse Rendering (CVPR 2026 · 0 citations)
Earliest papers here
- Adaptive 3D Face Reconstruction From Unconstrained Photo Collections (CVPR 2016 · 152 citations)
- Automated 3D Face Reconstruction From Multiple Images Using Quality Measures (CVPR 2016 · 90 citations)
- Learning Detailed Face Reconstruction From a Single Image (CVPR 2017 · 376 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
FFHQ · FID (lower is better)
6 papers, 2021–2025 · one report since 2024, not better
| Year | Best so far | Set by |
|---|---|---|
| 2021 | 5.09 | Inverting Generative Adversarial Renderer for Face Reconstruction (CVPR 2021 · 26 citations) |
| 2023 | 3.48 | FaceLit: Neural 3D Relightable Faces (CVPR 2023 · 21 citations) |
What has been tried
80 papers here report 758 measured changes (a component added, removed or swapped, compared with a baseline): helped 80%, no clear effect 3%, hurt 16%, mixed 1%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| remove the emotion loss | AffectNet test · E-ACC | 0.64 -> 0.62 | Hurt | 3D Facial Expressions through Analysis-by-Neural-Synthesis (CVPR 2024 · 48 citations) |
| Fine-tune Lp3D on real data | CelebA-HQ cross-reenactment · CSIM | 0.548 -> 0.670 | Helped | VOODOO 3D: Volumetric Portrait Disentanglement For One-Shot 3D Head Reenactment (CVPR 2024 · 19 citations) |
| use PlaneDict instead of removing it | Ablation study · CSIM | 0.763 -> 0.911 | Helped | Learning Dense Correspondence for NeRF-Based Face Reenactment (AAAI 2024 · 13 citations) |
| replace the MiT-b5 backbone with MiT-b1 | Multiface · SSME | 2.58 -> 3.21 | Hurt | 3D Face Tracking from 2D Video through Iterative Dense UV to Image Flow (CVPR 2024 · 9 citations) |
| use multi-domain codebooks instead of a joint codebook | Captured test set, diffuse albedo · PSNR | 24.87 -> 31.62 | Helped | Monocular Identity-Conditioned Facial Reflectance Reconstruction (CVPR 2024 · 6 citations) |
| Add Lipschitz regularization during stage 2 | Facial geometry reconstruction · CD | Helped | Do Not DeepFake Me: Privacy-Preserving Neural 3D Head Reconstruction Without Sensitive Images (AAAI 2025 · 4 citations) | |
| Add eyeball mesh priors to the neural-field face representation | Face geometry around eyes · geometry reconstruction quality | Helped | High-Quality Facial Geometry and Appearance Capture at Home (CVPR 2024 · 4 citations) | |
| Add the 3D geometry prior | Balanced CelebAHQ Test · LPIPS | 0.4703 -> 0.4671 | Helped | Blind Face Restoration under Extreme Conditions: Leveraging 3D-2D Prior Fusion for Superior Structural and Texture Recovery (AAAI 2024 · 3 citations) |
| add per-capture fine-tuning and appearance refinement | 17 captures · LPIPS | 0.218 -> 0.187 | Helped | Total Selfie: Generating Full-Body Selfies (CVPR 2024 · 3 citations) |
| Add IdRank selection to generated shape samples | NoW validation · median MSE | 1.02 -> 0.98 | Helped | OFER: Occluded Face Expression Reconstruction (CVPR 2025 · 2 citations) |
| replace channel attention with self-attention in the appearance network | 3D face rendering comparison · RMSE | 0.0276 -> 0.0302 | Hurt | FreeUV: Ground-Truth-Free Realistic Facial UV Texture Recovery via Cross-Assembly Inference Strategy (CVPR 2025 · 1 citation) |
| Add facial template | Facescape, ablation subset · PSNR | 24.27 -> 25.65 | Helped | SFDM: Robust Decomposition of Geometry and Reflectance for Realistic Face Rendering from Sparse-view Images (CVPR 2025 · 1 citation) |
What reviewers will expect
Datasets and baselines named in the 14 papers of this approach from 2024–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- FFHQ · 7 of 14 papers
- CelebA-HQ · 3 of 14 papers
- FaceScape · 3 of 14 papers
- FaMoS · 3 of 14 papers
- CelebAMask-HQ · 3 of 14 papers
- CelebA · 2 of 14 papers
- VoxCeleb2 · 2 of 14 papers
- H3DS · 2 of 14 papers
Baselines
- DECA · 3 of 14 papers
- ROME · 2 of 14 papers
- EMOCA · 2 of 14 papers
- HRN · 2 of 14 papers
- 3DDFAv2 · 2 of 14 papers
- MICA · 2 of 14 papers
- TokenFace · 2 of 14 papers
- MGCNet · 2 of 14 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 34 papers; from OpenAlex and the papers' affiliations).
- Tencent · 4 papers
- Nanyang Technological University · 4 papers
- Tsinghua University · 3 papers
- Hong Kong University of Science and Technology · 2 papers
- Chinese Academy of Sciences · 2 papers
- University of Chinese Academy of Sciences · 2 papers
- SenseTime · 2 papers
- Ant Group · 2 papers
- Shanghai Jiao Tong University · 2 papers
- University of British Columbia · 2 papers
Use it
Download BibTeX of 39 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- 3D morphable face model regression (80 papers)
- Speech-driven 3D facial animation (32 papers)
Elsewhere in the topic
- Neural implicit surface reconstruction (131 papers)
- Gaussian and implicit head avatars (115 papers)
- Textured mesh generation (111 papers)
- Primitive-based shape abstraction (99 papers)
- Single-view 3D reconstruction (96 papers)
- Full-body avatar reconstruction (78 papers)
- Articulated and non-rigid shape learning (75 papers)
- 3D shape generative modeling (74 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).