Research map / 3D vision / 3D generation
Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
placeholder
Problem
Creating realistic 3D human avatars from a single RGB image is hard: loose clothing and occlusions make monocular reconstruction ambiguous, and multi-view 2D diffusion priors lack guaranteed 3D consistency while deterministic reconstruction methods produce blurry unseen regions.
Approach
Diffuse multi-view renderings of a generative 3D-GS model that is conditioned on the 2D multi-view diffusion one-step estimate, train both jointly, and at each reverse step replace the 2D clean-image estimate with renderings of the predicted 3D-GS so the 2D sampling trajectory stays 3D consistent.
Key results
- On Sizer/IIIT/CAPE, Human-3Diffusion reports CD 1.41 cm and F-score 0.557 versus 2.75 cm and 0.359 for the best prior baseline (PIFu).
- Trajectory refinement improves multi-view quality on MVD: LPIPS 0.048 and PSNR 24.69 versus 0.078 and 22.32 for pretrained MVD.
- Removing trajectory refinement lowers final 3D reconstruction F-score from 0.550 to 0.498 and raises CD from 1.35 to 1.57 cm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Human-3Diffusion instead of prior reconstruction methods | Single-image avatar reconstruction · CAPE, Sizer, IIIT | CD (cm): 2.75 -> 1.41 | Helped |
| Use Human-3Diffusion instead of prior reconstruction methods | Single-image avatar reconstruction · CAPE, Sizer, IIIT | F-score: 0.359 -> 0.557 | Helped |
| Use Human-3Diffusion instead of prior reconstruction methods | Single-image avatar reconstruction · CAPE, Sizer, IIIT | LPIPS: 0.073 -> 0.058 | Helped |
| Add explicit 3D trajectory refinement to 2D multi-view sampling (ablation) | Human-3Diffusion · Human reconstruction | CD (cm): 1.57 -> 1.35 | Helped |
| Add explicit 3D trajectory refinement to 2D multi-view sampling (ablation) | Human-3Diffusion · Human reconstruction | PSNR: 21.09 -> 21.49 | Helped |
| Add explicit 3D trajectory refinement to 2D multi-view sampling (ablation) | 2D multi-view diffusion · Multi-view image generation | LPIPS: 0.078 -> 0.048 | Helped |
| Add the 2D multi-view prior (ablation) | Human-3Diffusion 3D generator · Human reconstruction | PSNR: 20.98 -> 21.49 | Helped |
| Add the 2D multi-view prior (ablation) | Human-3Diffusion 3D generator · GSO general objects | FID | Helped |
Surprising findings
- Fine-tuning LGM on human scans (LGM_human) does not fix its issues and yields a low normal consistency of 0.560.
Limitations
- Limited by 256x256 resolution of the multi-view diffusion model, so fine texture details like text are lost.
- Struggles with challenging poses, where head geometry and appearance are inaccurate.
- Evaluated on relatively few subjects (150 scans each from Sizer and IIIT), and training data is mostly casual clothing without object interaction.
Experimental setup
Datasets: 2k2k CustomHuman Thuman2.0 Thuman3.0 AXYZ Treedy Twindom RenderPeople
Benchmarks: Sizer IIIT-Human CAPE GSO
Metrics: Chamfer Distance P2S F-score Normal Consistency SSIM LPIPS PSNR FID
Baselines: PIFu SiTH SIFU LGM_human ICON ECON FoF TripoSR InstantMesh
Builds on: ImageDream 3D Gaussian Splatting DDPM LGM
Closest papers by meaning
- Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation (CVPR 2024 · similarity 0.92)
- DiffHuman: Probabilistic Photorealistic 3D Reconstruction of Humans (CVPR 2024 · similarity 0.91)
- GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion (CVPR 2025 · similarity 0.90)
- 3D²-Actor: Learning Pose-Conditioned 3D-Aware Denoiser for Realistic Gaussian Avatar Modeling (AAAI 2025 · similarity 0.90)
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (CVPR 2024 · similarity 0.89)
- En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.