Research map / 3D vision / Face and shape
GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds GeneMAN, a template-free single-image 3D human reconstruction framework using human-specific 2D (fine-tuned Stable Diffusion 1.5) and 3D (fine-tuned Zero-1-to-3) diffusion priors trained on a multi-source human dataset, with NeRF-to-DMTet geometry and multi-space (latent + pixel) texture refinement.
Problem
Reconstructing high-fidelity 3D human models from a single in-the-wild photo is hard due to varying body proportions, carried objects, ambiguous poses and textures, and scarce high-quality human 3D data.
Approach
Replace SMPL templates with human-specific 2D and 3D diffusion priors fine-tuned on a multi-source human dataset (3D scans, multi-view videos, single photos, and ControlNet/cropping-synthesized data), combined with an Initialization-and-Sculpting geometry pipeline and latent-then-pixel texture refinement.
Key results
- On in-the-wild images GeneMAN reaches PSNR 32.238, LPIPS 0.013, CLIP-Sim 0.730, beating PIFu (26.968, 0.035, 0.594) and TeCH.
- User study with 40 participants: 73.08% preferred GeneMAN over baselines.
- Geometry on CAPE (inference without GT pose): CD 1.8862 vs next-best SiTH 1.9182.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GeneMAN instead of PIFu | single-image 3D human reconstruction · in-the-wild | PSNR: 26.968 -> 32.238 | Helped |
| use GeneMAN instead of PIFu | single-image 3D human reconstruction · in-the-wild | LPIPS: 0.035 -> 0.013 | Helped |
| use GeneMAN instead of TeCH | single-image 3D human reconstruction · in-the-wild | CLIP-Sim: 0.713 -> 0.730 | Helped |
| use GeneMAN instead of GTA | single-image 3D human reconstruction · CAPE | PSNR: 30.376 -> 28.490 | Hurt |
| use GeneMAN instead of GTA | single-image 3D human reconstruction · CAPE | LPIPS: 0.019 -> 0.015 | Helped |
| add 3D scanned human data to the original prior models (ablation) | GeneMAN prior models · 30 test cases | CLIP-Sim: +0.046 CLIP-similarity | Helped |
| add 3D scanned human data to the original prior models (ablation) | GeneMAN prior models · 30 test cases | PSNR: 31.503 -> 30.873 | Hurt |
| add video human data to prior models trained on 3D scans (ablation) | GeneMAN prior models · 30 test cases | LPIPS: 0.017 -> 0.015 | Helped |
| add video human data to prior models trained on 3D scans (ablation) | GeneMAN prior models · 30 test cases | CLIP-Sim: 0.708 -> 0.713 | Helped |
| add augmented human data to prior models trained on 3D scans and video (ablation) | GeneMAN prior models · 30 test cases | CLIP-Sim: 0.713 -> 0.722 | Helped |
| add augmented human data to prior models trained on 3D scans and video (ablation) | GeneMAN prior models · 30 test cases | PSNR: 31.326 -> 31.205 | Hurt |
| add 2D human data to prior models trained on 3D scans, video, and augmented data (ablation) | GeneMAN prior models · 30 test cases | PSNR: 31.205 -> 32.238 | Helped |
Surprising findings
- Template-based baselines degrade substantially when GT SMPL pose is withheld, so the original CAPE comparisons were unfair to template-free methods.
- On CAPE, GTA achieves the best PSNR (30.376), above GeneMAN (28.490).
Limitations
- Optimization-based, requiring about 1.42 hours per asset, much slower than feed-forward methods.
- Poor fine-grained modeling of hands and no specific design for occluded individuals.
- Performs poorly with large or complex objects such as bicycles.
Experimental setup
Datasets: RenderPeople CustomHumans HuMMan THuman2.0 THuman3.0 X-Humans Objaverse DNA-Rendering ZJU-Mocap AIST++ Neural Actor Actors-HQ
Benchmarks: in-the-wild internet images CAPE
Metrics: PSNR LPIPS CLIP similarity Chamfer distance P2S normal consistency
Baselines: PIFu GTA TeCH SiTH PaMIR ICON ECON SIFU HumanLRM IDOL
Builds on: Zero-1-to-3 HumanNorm DreamFusion DMTet Instant-NGP
Closest papers by meaning
- MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinement (AAAI 2025 · similarity 0.90)
- PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing (CVPR 2025 · similarity 0.89)
- SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion (CVPR 2024 · similarity 0.89)
- SyncHuman: Synchronizing 2D and 3D Generative Models for Single-view Human Reconstruction (NeurIPS 2025 · similarity 0.88)
- HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion (ICLR 2024 · similarity 0.88)
- GenHMR: Generative Human Mesh Recovery (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.