Research map / 3D vision / 3D generation
AvatarVerse: High-Quality & Stable 3D Avatar Creation from Text and Pose
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates high-quality, pose-controlled 3D avatars from text and pose by using a DensePose-conditioned ControlNet inside SDS-based explicit NeRF optimization, with progressive high-resolution strategies and surface smoothing.
Problem
Text-to-3D avatar generation with SDS suffers from the Janus (multi-face) problem, view inconsistency, blurry/coarse geometry and missing fine details, and lacks precise pose control.
Approach
Condition the SDS guidance on a DensePose-conditioned ControlNet (trained on DeepFashion annotated with DensePose) so that pose and view correspondence are enforced, combined with progressive grid, progressive radius, focus mode, mesh refinement, and a density-gradient smoothness loss.
Key results
- In a user study with 16 volunteers on 30 generated outcomes, AvatarVerse was significantly preferred over DreamFusion, DreamAvatar, and DreamWaltz.
- Against DreamHuman on realistic humans, 81% of volunteers voted for AvatarVerse.
- Generation takes around 2 hours per prompt on a single NVIDIA A100 GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add progressive grid (ablation) | AvatarVerse · Progressive-strategy ablation | visual detail | Helped |
| Add progressive radius (ablation) | AvatarVerse with progressive grid · Progressive-strategy ablation | local detail | Helped |
| Add progressive radius (ablation) | AvatarVerse with progressive grid · Progressive-strategy ablation | floating artifacts | Helped |
| Add focus mode (ablation) | AvatarVerse with progressive grid and radius · Progressive-strategy ablation | local detail | Helped |
| Add mesh refinement (ablation) | AvatarVerse with progressive grid, radius, and focus mode · Progressive-strategy ablation | texture detail | Helped |
| Replace no additional control with skeleton control (ablation) | AvatarVerse · Control-signal ablation | human resemblance | Helped |
| Replace no additional control with skeleton control (ablation) | AvatarVerse · Control-signal ablation | view consistency | Hurt |
| Use DensePose control instead of skeleton control (ablation) | AvatarVerse · Control-signal ablation | avatar quality and view consistency | Helped |
| Add surface smoothing (ablation) | AvatarVerse · Surface-smoothing ablation | visual quality and realism | Helped |
| Use AvatarVerse instead of DreamHuman | AvatarVerse · User study: realistic humans | volunteer preference: 81% | Helped |
Limitations
- Relies on qualitative comparisons and a single user study with 16 volunteers; no standard quantitative metrics reported in the main text.
- Local artifacts may remain when relying solely on the DensePose-conditioned ControlNet, requiring progressive refinement.
- Generation takes about 2 hours per avatar on an A100.
Experimental setup
Datasets: DeepFashion DensePose-COCO
Metrics: user preference votes
Baselines: DreamFusion DreamAvatar DreamWaltz DreamHuman Avatar-CLIP AvatarCraft HeadSculpt ProlificDreamer
Builds on: DreamFusion ControlNet Latent-NeRF SMPL DensePose
Closest papers by meaning
- DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models (CVPR 2024 · similarity 0.90)
- Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation (CVPR 2024 · similarity 0.89)
- X-Oscar: A Progressive Framework for High-quality Text-guided 3D Animatable Avatar Generation (ICML 2024 · similarity 0.89)
- UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with Authenticity Guided Textures (CVPR 2024 · similarity 0.88)
- En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data (CVPR 2024 · similarity 0.88)
- Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.