Research map / 3D vision / 3D generation
GaussianIP: Identity-Preserving Realistic 3D Human Generation via Human-Centric Diffusion Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GaussianIP, a two-stage 3D Gaussian Splatting framework that generates identity-preserving, detailed 3D humans from text plus a face image using human-centric diffusion distillation (AHDS) and a view-consistent diffusion refinement (VCR).
Problem
Text-guided 3D human generation is slow to train and lacks fine facial and garment detail, and existing methods cannot take image prompts to preserve a user's identity.
Approach
Distill a human-centric identity-conditioned diffusion prior (IP-Adapter-FaceID-PlusV2 with pose ControlNet) using a decomposed score difference (HDS) and a three-phase adaptive timestep schedule (AHDS), then refine multi-view renders with mutual attention and distance-guided attention fusion before reconstructing the 3DGS avatar.
Key results
- AHDS reduces HDS training steps from 3600 to 2400 (about 30%) while improving visual quality over plain SDS.
- GaussianIP reaches about 40 minutes of training versus over one hour for baselines, with Face++ identity match confidence above 83%.
- In the user study, GaussianIP scores 4.71 on facial detail and 4.50 on clothing texture richness, above HumanGaussian's 4.29 and 4.17.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GaussianIP instead of HumanGaussian | 3D human generation · User study, 20 prompts | Facial Detail: 4.29 -> 4.71 | Helped |
| Use GaussianIP instead of HumanGaussian | 3D human generation · User study, 20 prompts | Clothing Texture Richness: 4.17 -> 4.50 | Helped |
| Use GaussianIP instead of HumanGaussian | 3D human generation · User study, 20 prompts | Overall Visual Quality: 3.96 -> 4.17 | Helped |
| Use GaussianIP instead of HumanGaussian | 3D human generation · User study, 20 prompts | Text Prompt Alignment: 4.42 -> 4.62 | Helped |
| Use GaussianIP instead of HumanGaussian | 3D human generation · User study, 20 prompts | GPT Score: 4.08 -> 4.52 | Helped |
| Use GaussianIP instead of HumanGaussian | 3D human generation · Text-guided 3D human generation | Training time: 1.2h -> 40min | Helped |
| Add Human Distillation Sampling (HDS) to the 3DGS+SDS baseline (ablation) | GaussianIP stage 1 · Text- and image-guided 3D human generation | Identity preservation and clothing details | Helped |
| Add adaptive timestep scheduling to HDS (ablation) | GaussianIP stage 1 · Text- and image-guided 3D human generation | Training steps: 3600 -> 2400 | Helped |
| Add adaptive timestep scheduling to HDS (ablation) | GaussianIP stage 1 · Text- and image-guided 3D human generation | Generative quality | Helped |
| Add View-Consistent Refinement (VCR) (ablation) | GaussianIP stage 2 · Multi-view 3D human generation | Cross-view texture consistency | Helped |
| Denoise each view independently instead of using VCR (ablation) | GaussianIP stage 2 · Multi-view 3D human generation | Cross-view texture consistency | Hurt |
Limitations
- Struggles with highly complex poses or extreme garment textures
- Does not yet handle human-object or human-human interactions
Experimental setup
Benchmarks: user study on 20 text prompts
Metrics: facial detail score clothing texture richness overall visual quality text prompt alignment GPT score training time Face++ identity confidence
Baselines: DreamWaltz TADA AvatarVerse X-Oscar HumanGaussian
Builds on: DreamFusion HumanGaussian 3D Gaussian Splatting IP-Adapter-FaceID-PlusV2 ControlNet
Closest papers by meaning
- HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting (CVPR 2024 · similarity 0.92)
- HumanRef: Single Image to 3D Human Generation via Reference-Guided Diffusion (CVPR 2024 · similarity 0.89)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.88)
- Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models (NeurIPS 2024 · similarity 0.88)
- GauHuman: Articulated Gaussian Splatting from Monocular Human Videos (CVPR 2024 · similarity 0.88)
- HUGS: Human Gaussian Splats (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.