Research map / 3D vision / Face and shape
DiffHuman: Probabilistic Photorealistic 3D Reconstruction of Humans
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffHuman, a conditional diffusion model over pixel-aligned front/back albedo, normal and depth renders of an implicit human surface, with a generator network that replaces costly rendering in most denoising steps for a 55x speedup.
Problem
Monocular 3D human reconstruction from a single RGB image is ill-posed, and deterministic methods output a single blurry solution that regresses to the mean in unseen or uncertain regions.
Approach
Model a distribution over photorealistic 3D human reconstructions by running a DDPM over pixel-aligned observations of an implicit surface, rendering the intermediate SDF in denoising, and training a generator U-Net to imitate rendering so that sampling is much faster (hybrid implicit surface diffusion).
Key results
- Generator-based denoising gives up to 55x speed-up (9 s vs 496 s per sample) at comparable quality to per-step rendering.
- With best-of-10 samples, DiffHuman achieves Chamfer 1.09, IoU 0.73, NC 0.86, roughly matching retrained PHORHUM (1.10, 0.73, 0.87).
- Back-side albedo PSNR improves from 19.77 (N=1) to 21.24 (N=10), showing gains in unseen regions with more samples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use best of 10 samples instead of 1 (ablation) | DiffHuman · Test set of [5] | Albedo back PSNR: 19.77 -> 21.24 | Helped |
| use best of 10 samples instead of 1 (ablation) | DiffHuman · Test set of [5] | Chamfer distance ×10⁻³: 1.98 -> 1.09 | Helped |
| use best of 10 samples instead of 1 (ablation) | DiffHuman · Test set of [5] | IoU: 0.69 -> 0.73 | Helped |
| render only at the final denoising step instead of every step (ablation) | DiffHuman hybrid diffusion · Test set of [5] | runtime (s / sample): 496 -> 9 | Helped |
| render only at the final denoising step instead of every step (ablation) | DiffHuman hybrid diffusion · Test set of [5] | Chamfer distance ×10⁻³: 0.99 -> 1.16 | Hurt |
| render only at the final denoising step instead of every step (ablation) | DiffHuman hybrid diffusion · Test set of [5] | Albedo front PSNR: 22.92 -> 23.26 | Helped |
| use DiffHuman instead of retrained PHORHUM | single-image human reconstruction · Test set of [5] | Shaded front PSNR: 25.23 -> 26.81 | Helped |
| use DiffHuman with best of 10 samples instead of retrained PHORHUM | single-image human reconstruction · Test set of [5] | Albedo back PSNR: 19.74 -> 21.24 | Helped |
Surprising findings
- Retrained PHORHUM, a deterministic method, is a very strong baseline and often performs slightly better than DiffHuman with N=1 on 3D metrics.
Limitations
- Requires training examples with known 3D geometry, limiting usable data.
- Best-of-N metrics favour sampling; N=1 is weaker than deterministic baselines on 3D metrics.
- Albedo predictions show shading artefacts due to imperfect ambient-lit capture.
Experimental setup
Datasets: synthetic renders of real body scans from Renderpeople-style scan dataset [45] in-house captured scans PHORHUM test set
Benchmarks: PHORHUM test set
Metrics: bi-directional Chamfer distance normal consistency volumetric IoU SSIM LPIPS PSNR normal angular error
Baselines: PHORHUM S3F PIFu PIFuHD ICON ECON ARCH ARCH++ PaMIR Geo-PIFu D-IF
Builds on: DDPM PHORHUM PIFu diffusion-via-rendering (RenderDiffusion/HoloDiffusion) DDIM
Closest papers by meaning
- Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models (NeurIPS 2024 · similarity 0.91)
- PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing (CVPR 2025 · similarity 0.91)
- SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion (CVPR 2024 · similarity 0.89)
- DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesis (CVPR 2024 · similarity 0.89)
- Coherent 3D Scene Diffusion From a Single RGB Image (NeurIPS 2024 · similarity 0.89)
- SyncHuman: Synchronizing 2D and 3D Generative Models for Single-view Human Reconstruction (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.