Research map / 3D vision / Face and shape
PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
PSHuman generates six multiview color and normal images of a clothed person from one photo using a SMPL-X-conditioned, body-face cross-scale diffusion model, then carves a textured mesh initialized from SMPL-X via differentiable remeshing.
Problem
Monocular full-body 3D human reconstruction from a single image suffers from self-occlusion, ill-posed depth, and distorted faces, so existing methods that rely on front and/or predicted back views struggle with geometry and appearance quality.
Approach
Fine-tune a multiview diffusion model to generate six full-body views conditioned on rendered SMPL-X, jointly denoise a high-resolution face branch via a noise blending layer to preserve facial identity, and reconstruct the mesh explicitly by remeshing SMPL-X against generated normal maps and fusing generated color images for texture.
Key results
- On CAPE-NFP with ground-truth SMPL-X, PSHuman reaches Chamfer 0.9688 cm vs 1.5511 for ICON and 1.8118 for SiTH (w/ SMPL-X prior).
- Appearance on THuman2.1 subset: PSNR 20.85 vs 19.61 for GTA and LPIPS 0.0764 vs 0.0985 for SiFU.
- Face reconstruction Chamfer 0.423 vs 0.535 for SiFU; full pipeline takes about one minute.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SMPL-X conditioning to the diffusion model (ablation) | PSHuman · CAPE-NFP subset | Chamfer distance: 1.4370 -> 1.0938 | Helped |
| add cross-scale diffusion (ablation) | PSHuman · CAPE-NFP subset | Chamfer distance: 1.0938 -> 0.9597 | Helped |
| use SMPLX-guided differentiable remeshing instead of ECON reconstruction (ablation) | PSHuman · CAPE-NFP subset | Chamfer distance: 1.2630 -> 0.9597 | Helped |
| add the local face branch (ablation) | PSHuman cross-scale diffusion · Face reconstruction | PSNR: 19.67 -> 20.97 | Helped |
| add noise blending between face and body branches (ablation) | PSHuman cross-scale diffusion · Face reconstruction | PSNR: 20.85 -> 20.97 | Helped |
| add SMPL-X-conditioned multiview diffusion (ablation) | PSHuman · Complex posed humans with self-occlusion | cross-view body-shape consistency | Helped |
| use explicit remeshing instead of NeuS or BiNI (ablation) | PSHuman carving module · Human mesh reconstruction | geometry detail | Helped |
| add multiview-guided SMPL-X alignment before remeshing (ablation) | PSHuman carving module · Human mesh reconstruction | geometry quality | Helped |
| add noise to SMPL-X pose and shape parameters (ablation) | PSHuman and template-based baselines · THuman2.1 subset | Chamfer distance: 0.4399 -> 0.5574 | Mixed |
Surprising findings
- Directly applying multiview diffusion to single-view human images causes severe geometric distortion, especially on faces.
Limitations
- Pose estimation errors cascade into generation and reconstruction.
- Wrong novel-view generation can produce unreasonable geometry.
- Diffusion struggles with fine details such as loose hair and hands.
Experimental setup
Datasets: THuman2.1 CustomHumans CAPE
Benchmarks: THuman2.1 CAPE
Metrics: Chamfer distance P2S normal consistency PSNR SSIM LPIPS
Baselines: PIFu PIFuHD PaMIR ICON ECON GTA SiFU HiLo SiTH
Builds on: Stable Diffusion 2.1-unclip ICON ECON SiTH Wonder3D-style multiview diffusion
Closest papers by meaning
- SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion (CVPR 2024 · similarity 0.91)
- SyncHuman: Synchronizing 2D and 3D Generative Models for Single-view Human Reconstruction (NeurIPS 2025 · similarity 0.91)
- DiffHuman: Probabilistic Photorealistic 3D Reconstruction of Humans (CVPR 2024 · similarity 0.91)
- MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinement (AAAI 2025 · similarity 0.89)
- GeneMAN: Generalizable Single-Image 3D Human Reconstruction from Multi-Source Human Data (NeurIPS 2025 · similarity 0.89)
- HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.