Research map / 3D vision / 3D generation
ID-Sculpt: ID-aware 3D Head Generation from Single In-the-wild Portrait Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ID-Sculpt, an optimization-based framework that injects identity information from a single portrait into geometry initialization (3D-GAN inversion), geometry sculpting (ID-aware score distillation), and texture generation to produce 3D heads.
Problem
Generating high-fidelity 3D human heads with accurate geometry and identity-preserving, photorealistic texture from a single in-the-wild portrait image.
Approach
Integrates identity features (ID adapter with decoupled cross-attention), landmark-guided ControlNet, and canny-conditioned text-to-normal/depth diffusion into every stage: 3D-GAN inversion initialization, ID-aware Score Distillation for geometry, and ID-consistent progressive texture inpainting plus diffusion-based refinement.
Key results
- ID-Sculpt achieves the best ID score (0.4451) and LPIPS (0.039) among compared methods, versus PanoHead's 0.2737 ID and 0.045 LPIPS.
- In a user study over 480 responses from 32 participants, ID-Sculpt ranked highest with an average score of 4.72 versus PanoHead's 4.09.
- Ablation shows removing ID-aware initialization drops ID score from 0.4451 to 0.2296 and raises LPIPS to 0.218.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ID-aware initialization (ablation) | ID-Sculpt · FFHQ (106 portraits) | ID: 0.2296 -> 0.4451 | Helped |
| add ID-aware guidance (ablation) | ID-Sculpt · FFHQ (106 portraits) | ID: 0.2984 -> 0.4451 | Helped |
| add ID-aware geometry supervision (ablation) | ID-Sculpt · FFHQ (106 portraits) | ID: 0.4337 -> 0.4451 | Helped |
| add texture refinement (ablation) | ID-Sculpt · FFHQ (106 portraits) | ID: 0.3965 -> 0.4451 | Helped |
| add progressive texture inpainting (ablation) | ID-Sculpt · FFHQ (106 portraits) | ID: 0.4269 -> 0.4451 | Helped |
| replace ID-aware normal supervision with DPT normals (ablation) | ID-Sculpt geometry sculpting · 3D head generation | facial geometry detail | Hurt |
| replace ID-aware depth and normal maps with DPT predictions (ablation) | ID-Sculpt geometry sculpting · 3D head generation | geometry detail | Hurt |
| use only ID-aware depth supervision (ablation) | ID-Sculpt geometry sculpting · 3D head generation | geometry detail | Hurt |
| replace ISD loss with SDS loss (ablation) | ID-Sculpt geometry sculpting · 3D head generation | geometry quality | Hurt |
| use FLAME as initialization geometry (ablation) | ID-Sculpt geometry initialization · 3D head generation | geometry alignment | Hurt |
| remove ID-aware guidance enhancement during texture generation (ablation) | ID-Sculpt texture generation · large-angle views | identity consistency | Hurt |
| remove texture refinement and rely only on inpainting (ablation) | ID-Sculpt texture generation · 3D head texture | texture boundary continuity | Hurt |
Limitations
- Optimization takes about 40 minutes per head, which is slow.
- Evaluated on only 106 FFHQ portraits with a user study of 15 examples.
Experimental setup
Datasets: FFHQ
Benchmarks: FFHQ subset (106 portraits)
Metrics: LPIPS CLIP-I ID score user study
Baselines: Magic123 DreamCraft3D Wonder3D PanoHead HumanNorm
Builds on: DreamFusion ProlificDreamer Zero-1-to-3 PanoHead IP-Adapter
Closest papers by meaning
- ID-to-3D: Expressive ID-guided 3D Heads via Score Distillation Sampling (NeurIPS 2024 · similarity 0.90)
- Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior (CVPR 2024 · similarity 0.87)
- En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data (CVPR 2024 · similarity 0.86)
- Image Sculpting: Precise Object Editing with 3D Geometry Control (CVPR 2024 · similarity 0.86)
- Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance (CVPR 2025 · similarity 0.86)
- Text-Guided 3D Face Synthesis - From Generation to Editing (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.