Research map / 3D vision / Face and shape
Do Not DeepFake Me: Privacy-Preserving Neural 3D Head Reconstruction Without Sensitive Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a two-stage neural 3D head reconstruction method that first uses non-sensitive rear-head RGB images and then refines geometry using only gradient-magnitude images of privacy-protected frontal faces, achieving geometry comparable to VolSDF with full images while resisting FR and DeepFake.
Problem
Neural 3D head reconstruction (NeRF/SDF-based) typically requires high-resolution multi-view facial images, which pose serious privacy risks such as unauthorized tracking and DeepFake creation.
Approach
Reconstruct the head in two stages: stage 1 trains a VolSDF-style SDF model on privacy-neutral rear-head RGB images for coarse geometry; stage 2 refines it using only image gradient magnitudes (e.g., Sobel) of privacy-protected frontal images, with a gradient loss and a Lipschitz regularization term, so that no sensitive RGB facial texture is used.
Key results
- Stage 2 Chamfer distance on FaceScape averages 0.7824e-3 versus 0.4948e-3 for VolSDF with full images; on H3DS it averages 3.0477 mm versus 2.6351 mm.
- Face recognition accuracy across 7 systems drops from 91.67% on original images to 6.05% on protected images and 33.95% on the authors' renderings (VolSDF renderings: 77.64%).
- The paper reports that the reconstruction is comparable to full-image VolSDF while the protected inputs resist FR and DeepFake.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Refine stage-1 geometry using privacy-protected gradient images in stage 2 | Ours two-stage reconstruction · FaceScape | CD ($10^{-3}$): 2.3552 -> 0.7824 | Helped |
| Refine stage-1 geometry using privacy-protected gradient images in stage 2 | Ours two-stage reconstruction · H3DS | CD (mm): 4.7066 -> 3.0477 | Helped |
| Use the two-stage privacy-preserving method instead of full-image VolSDF | Head reconstruction · FaceScape | CD ($10^{-3}$): 0.4948 -> 0.7824 | Hurt |
| Use the two-stage privacy-preserving method instead of full-image VolSDF | Head reconstruction · H3DS | CD (mm): 2.6351 -> 3.0477 | Hurt |
| Add Lipschitz regularization during stage 2 (ablation) | Ours two-stage reconstruction · Facial geometry reconstruction | CD | Helped |
| Remove the stage-1 geometric-prior initialization and reconstruct from gradient examples only (ablation) | Ours reconstruction · Facial geometry reconstruction | Geometry quality | Hurt |
| Train in one stage with both RGB and gradient losses on a template (ablation) | Ours reconstruction · Facial geometry reconstruction | Geometry quality | Hurt |
| Use a neutral head template when privacy-neutral images are scarce (ablation) | Ours reconstruction · Facial geometry reconstruction | Geometry quality | Hurt |
| Vary the number of RGB images and views used in stage 1 (ablation) | Ours two-stage reconstruction · Facial geometry reconstruction | Final reconstruction quality | No clear effect |
Surprising findings
- Stage 2 geometry from gradient-only images is roughly comparable to full-RGB VolSDF even though the gradient inputs discard most facial texture.
- Protected-image and rendered-mesh face-recognition accuracy falls to around 6%, while the rendered-mesh results contain visible identity-free geometry.
Limitations
- Stage 2 geometry is less accurate than full-image VolSDF on both datasets (e.g., mean CD 0.78 vs 0.49 on FaceScape).
- Experiments use only 10 identities per dataset at 64x64 resolution.
- Without stage 1 or with templates, results are less satisfactory; template-only geometry lacks low-frequency information.
Experimental setup
Datasets: FaceScape H3DS
Benchmarks: FaceScape H3DS
Metrics: Chamfer distance cosine distance (face recognition) LPIPS recognition accuracy
Baselines: VolSDF (full images)
Builds on: VolSDF NeuS NeRF Lipschitz MLP regularization
Closest papers by meaning
- Deep 3D Portrait From a Single Image (CVPR 2020 · similarity 0.86)
- Unsupervised Depth Estimation, 3D Face Rotation and Replacement (NeurIPS 2018 · similarity 0.85)
- ID-to-3D: Expressive ID-guided 3D Heads via Score Distillation Sampling (NeurIPS 2024 · similarity 0.84)
- DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion (NeurIPS 2024 · similarity 0.84)
- Gaussian Differentially Private Human Faces Under a Face Radial Curve Representation (ICLR 2025 · similarity 0.83)
- VOODOO 3D: Volumetric Portrait Disentanglement For One-Shot 3D Head Reenactment (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.