Research map / 3D vision / Face and shape
SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
SiTH reconstructs fully textured 3D human meshes from a single image by hallucinating the back view with an image-conditioned latent diffusion model (ControlNet-style, UV and mask conditioned) and then running a skinned-body-guided implicit SDF/RGB reconstruction from front and back views, trained on 500 THuman2.0 scans.
Problem
Reconstructing fully textured, detailed 3D human meshes from a single-view image is ill-posed: appearance, depth, and clothing details in unseen (back) regions are missing, and data-driven methods trained on limited 3D scans fail on unseen appearances and poses.
Approach
Decompose single-view reconstruction into generative back-view hallucination (an LDM fine-tuned with image conditioning via CLIP and VAE features plus ControlNet UV and silhouette-mask conditions) and mesh reconstruction (pixel-aligned SDF/RGB features from front and generated back normals/images, with local positional embedding relative to a skinned SMPL-X body mesh).
Key results
- On CustomHumans, SiTH achieves CD 1.958 cm, NC 0.826, f-score 37.029, competitive with methods trained on commercial data.
- In a 30-user study, SiTH was preferred over Chupa in 64.0% of comparisons.
- SiTH generates a textured mesh in under two minutes, versus about 10 minutes for Zero-1-to-3 and 6 hours for Magic123 optimization.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SiTH instead of the strongest reported baseline, PaMIR | single-view human reconstruction · CAPE | CD: P-to-S / S-to-P (cm): 2.190 / 2.806 -> 1.899 / 2.261 | Helped |
| use SiTH instead of the strongest reported baseline, PaMIR | single-view human reconstruction · CustomHumans | LPIPS: F / B (×10⁻²): 4.646 / 7.152 -> 3.929 / 6.803 | Helped |
| add skinned body mesh guidance (ablation) | SiTH · CustomHumans | f-Score: 33.244 -> 37.029 | Helped |
| add back-view hallucination module (ablation) | SiTH · CustomHumans | f-Score: 36.677 -> 37.029 | Helped |
| use a ground-truth body mesh (ablation) | SiTH · CustomHumans | f-Score: 37.029 -> 58.858 | Helped |
| use a ground-truth body mesh and real back-view image (ablation) | SiTH · CustomHumans | f-Score: 37.029 -> 63.356 | Helped |
| remove back-view hallucination module (ablation) | SiTH · CustomHumans | NC: 0.826 -> 0.840 | Mixed |
Surprising findings
- Removing the back-view hallucination module has minimal effect on 3D metrics (CD 1.960 vs 1.958) yet visibly degrades texture and geometry quality, so 3D metrics may not reflect perceptual quality.
Limitations
- Single-view reconstruction cannot replicate exact back-view geometry and texture; stochastic clothing details are inherently ambiguous.
- Depends on an off-the-shelf pose predictor and segmentation model at test time, and on an SMPL-X fit.
Experimental setup
Datasets: THuman2.0 CAPE CustomHumans
Benchmarks: CAPE CustomHumans
Metrics: Chamfer distance normal consistency f-score LPIPS user preference ranking
Baselines: PIFu PIFuHD PaMIR 2K2K FOF ICON ECON PHORHUM Zero-1-to-3 Magic123 Chupa DreamPose
Builds on: Stable Diffusion ControlNet PIFu PIFuHD ICON PaMIR
Closest papers by meaning
- PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing (CVPR 2025 · similarity 0.91)
- SyncHuman: Synchronizing 2D and 3D Generative Models for Single-view Human Reconstruction (NeurIPS 2025 · similarity 0.91)
- DeClotH: Decomposable 3D Cloth and Human Body Reconstruction from a Single Image (CVPR 2025 · similarity 0.89)
- DiffHuman: Probabilistic Photorealistic 3D Reconstruction of Humans (CVPR 2024 · similarity 0.89)
- GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data (NeurIPS 2025 · similarity 0.89)
- Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.