Research map / 3D vision / Face and shape
ID-to-3D: Expressive ID-guided 3D Heads via Score Distillation Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates identity- and text-guided 3D human heads with disentangled expressions from 1-5 in-the-wild images using SDS with LoRA-adapted Stable Diffusion geometry/albedo priors conditioned on ArcFace and expression embeddings.
Problem
Generating expressive, identity-consistent 3D human heads from casually captured images without large 3D scan datasets; existing SDS methods suffer from Janus artifacts, poor texture/geometry separation, and identity drift.
Approach
Fine-tune Stable Diffusion with LoRA (about 0.2% of parameters) into separate normal-map and albedo-map 2D priors conditioned on ArcFace identity and CLIP expression embeddings via IP-Adapter, and optimize a DMTET-based neural parametric head with per-expression latent codes using SDS.
Key results
- ID-to-3D achieves the highest average CosFace identity similarity with the lowest variance across views versus Fantasia3D, HumanNorm, TADA, DreamFace, Magic123 and DreamCraft3D.
- ID-to-3D reports the lowest FID for both texture (vs. Stable Diffusion V1.5 renders) and geometry (vs. NHPM normals) and the highest user preference in a 50-participant study.
- The 2D guidance fine-tunes only about 0.2% of the trainable parameters, and expression-jointly trained geometry for 13 expressions takes about 4 hours on V100 GPUs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ID-to-3D instead of the compared text-to-3D and image-to-3D methods | 3D head generation · 40-celebrity benchmark | FID (geometry) | Helped |
| Use ID-to-3D instead of the compared text-to-3D and image-to-3D methods | 3D head generation · 40-celebrity benchmark | FID (texture) | Helped |
| Use ID-to-3D instead of the compared text-to-3D and image-to-3D methods | 3D head generation · 40-celebrity benchmark | user preference (geometry and texture) | Helped |
| Use ArcFace features collected from five images rather than identity conditioning from a single in-the-wild picture (ablation) | ID-to-3D · 3D head identity conditioning | identity similarity | Helped |
| Add randomized lighting during texture training (ablation) | ID-to-3D · Texture generation | texture detail and color quality | Helped |
Surprising findings
- DreamFace lacks hair and eyes by design, leading to the lowest average identity similarity among methods compared.
Limitations
- Generalization limited by the face embedding network and diffusion model, which may introduce representation biases.
- Albedo is only pseudo-albedo without physically bounded constraints, occasionally producing unnatural facial characteristics.
- Computational cost of PBR rendering hinders video-driven applications.
Experimental setup
Datasets: NHPM neural parametric head dataset BingImages (celebrity test images)
Benchmarks: 40 celebrity identity test set
Metrics: CosFace identity similarity FID user preference study
Baselines: Fantasia3D HumanNorm TADA DreamFace Magic123 DreamCraft3D
Builds on: DreamFusion Fantasia3D IP-Adapter LoRA DMTET
Closest papers by meaning
- ID-Sculpt: ID-aware 3D Head Generation from Single In-the-wild Portrait Image (AAAI 2025 · similarity 0.90)
- Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance (CVPR 2025 · similarity 0.89)
- DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars (CVPR 2024 · similarity 0.87)
- Data Synthesis with Diverse Styles for Face Recognition via 3DMM-Guided Diffusion (CVPR 2025 · similarity 0.86)
- 3D Facial Expressions through Analysis-by-Neural-Synthesis (CVPR 2024 · similarity 0.86)
- CapHuman: Capture Your Moments in Parallel Universes (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.