Research map / 3D vision / 3D generation
PaintHuman: Towards High-Fidelity Text-to-3D Human Texturing via Denoised Score Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PaintHuman, a zero-shot text-to-3D human texturing method that modifies SDS with a negative image-text pair (Denoised Score Distillation), adds depth-conditioned diffusion, and uses SV-BRDF shading for textured SMPL-like human meshes.
Problem
Score Distillation Sampling for text-driven human mesh texturing yields over-smoothed, low-detail textures that are misaligned with the text and with the detailed mesh geometry.
Approach
Denoised Score Distillation subtracts a weighted score term computed from a negative image (previous-iteration render) and negative text prompt from the SDS gradient, combined with depth-to-image guidance and learned SV-BRDF material prediction for geometry-aware rendering.
Key results
- Mean CLIP score of 28.93 versus 27.10 for Fantasia3D, 25.34 for TEXTure, and 24.11 for Latent-Paint.
- User study mean score 3.01 versus 2.83 for DreamHuman and 1.21 for Latent-Paint.
- Against DreamHuman's reported results, CLIP score 28.95 vs 25.79.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PaintHuman instead of Fantasia3D | Textured human avatars on RenderPeople · RenderPeople | mean CLIP score: 27.10 -> 28.93 | Helped |
| Use PaintHuman instead of DreamHuman | Textured human avatars in the DreamHuman comparison · DreamHuman comparison | mean CLIP score: 25.79 -> 28.95 | Helped |
| Use PaintHuman instead of DreamHuman | Human avatar texturing · User study | overall quality score: 2.83$\pm$0.82 -> 3.01$\pm$0.95 | Helped |
| Add DSD with a negative image-text pair instead of using SDS (ablation) | PaintHuman human texturing · Man in a suit with a belt and tie | texture detail and text alignment | Helped |
| Add depth map guidance to SDS (ablation) | Human texturing with SDS · Man in a suit with a belt and tie | text alignment | No clear effect |
| Add negative prompts without the full DSD setup (ablation) | Human texturing · Man in a suit with a belt and tie | texture detail and text alignment | Mixed |
| Replace BRDF shading with Spherical Harmonic shading (ablation) | PaintHuman rendering · Man in a suit with a belt and tie | texture realism | Hurt |
| Add semantic zoom on the face region (ablation) | PaintHuman · Human face region | facial texture quality | Helped |
Limitations
- DreamHuman comparison uses similar meshes and published figures because DreamHuman is not publicly available, so the comparison is not controlled.
- Negative prompts are tuned manually from test runs.
- Quantitative evaluation is limited to CLIP score and a small user study with 4 samples per baseline.
Experimental setup
Datasets: RenderPeople
Benchmarks: RenderPeople
Metrics: mean CLIP score user study score (1-5)
Baselines: Latent-Paint TEXTure Fantasia3D DreamHuman
Builds on: DreamFusion Fantasia3D Latent-NeRF TEXTure classifier-free guidance
Closest papers by meaning
- Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering (CVPR 2024 · similarity 0.89)
- HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting (CVPR 2024 · similarity 0.89)
- HumanNorm: Learning Normal Diffusion Model for High-quality and Realistic 3D Human Generation (CVPR 2024 · similarity 0.89)
- HIFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance (ICLR 2024 · similarity 0.88)
- HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion (ICLR 2024 · similarity 0.88)
- ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.