Research map / 3D vision / Face and shape
SFDM: Robust Decomposition of Geometry and Reflectance for Realistic Face Rendering from Sparse-view Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SFDM, a two-stage neural inverse-rendering framework that learns a facial template from multiple individuals and then refines per-person geometry, diffuse/specular reflectance, and subsurface scattering from as few as three views.
Problem
Decomposing face geometry, diffuse and specular reflectance, and lighting from sparse-view images is ill-posed; prior face decomposition (NeuFace) collapses to diffuse-only appearance and degrades geometry with fewer views.
Approach
Learn a general facial template (geometry plus BRDF with an albedo gradient predictor) across individuals in Stage 1, then refine per-individual geometry, BRDF offsets, learnable template weights, attribute synergy between geometry and reflectance, and a subsurface-scattering diffuse offset in Stage 2.
Key results
- At 3 views, SFDM reaches 25.54 PSNR / 0.902 SSIM / 4.46 LPIPS (x1e-2) / 9.27 CD (x1e-3), versus NeuFace 23.54 / 0.876 / 5.78 / 15.74 on FaceScape.
- On diffuse/specular SSIM against pseudo ground truth from RefMM, SFDM scores 0.761/0.692 versus NeuFace 0.726/0.552.
- Ablation: removing the facial template drops PSNR from 25.65 to 24.27 (with the full model at 25.65 in the ablation subset).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SFDM instead of NeuFace | Face reconstruction and decomposition · Facescape, 3 views | LPIPS (×10−2): 5.78 -> 4.46 | Helped |
| Use SFDM instead of NeuFace | Face reconstruction and decomposition · Facescape, 5 views | LPIPS (×10−2): 4.75 -> 3.83 | Helped |
| Use SFDM instead of NeuFace | Face reconstruction and decomposition · Facescape, 10 views | SSIM: 0.914 -> 0.920 | Helped |
| Add facial template (ablation) | SFDM · Facescape, ablation subset | PSNR: 24.27 -> 25.65 | Helped |
| Add albedo gradient predictor (ablation) | SFDM · Facescape, ablation subset | PSNR: 25.13 -> 25.65 | Helped |
| Add attribute synergy module (ablation) | SFDM · Facescape, ablation subset | CD (×10−3): 8.80 -> 7.94 | Helped |
| Add subsurface scattering offset (ablation) | SFDM · Facescape, ablation subset | LPIPS (×10−2): 5.73 -> 4.45 | Helped |
Surprising findings
- NeuFace often collapses to a degenerate Lambertian solution with no specular term under sparse views.
- VolSDF and DeformHead failed on 21.7% and 6.7% of reconstructions respectively, and these failures were excluded from comparison.
Limitations
- Method is specific to faces and hard to extend directly to general objects.
- No reflectance ground truth; diffuse/specular evaluation relies on coarse pseudo ground truth from a single-view model.
- Training is slow (3000 epochs, no hash encoding acceleration).
Experimental setup
Datasets: FaceScape H3DS
Benchmarks: FaceScape
Metrics: PSNR SSIM LPIPS Chamfer distance
Baselines: VolSDF DeformHead PhySG TensoIR NeuFace
Builds on: VolSDF NeuFace PhySG DeformHead split-sum pre-computed BRDF integration
Closest papers by meaning
- Monocular Identity-Conditioned Facial Reflectance Reconstruction (CVPR 2024 · similarity 0.88)
- S^3-Face: SSS-Compliant Facial Reflectance Estimation via Diffusion Priors (CVPR 2025 · similarity 0.87)
- 3D Facial Expressions through Analysis-by-Neural-Synthesis (CVPR 2024 · similarity 0.86)
- Low-Light Face Super-resolution via Illumination, Structure, and Texture Associated Representation (AAAI 2024 · similarity 0.86)
- Learning to Decouple the Lights for 3D Face Texture Modeling (NeurIPS 2024 · similarity 0.85)
- OFER: Occluded Face Expression Reconstruction (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.