Research map / 3D vision / Face and shape
Generalizable Human Gaussians from Single-View Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HGM, a single-view generalizable human Gaussian model that uses SMPL-X dual-branch priors, iterative SMPL-X refinement, and a ControlNet back-view refinement with diffusion priors to produce detailed human Gaussians.
Problem
Reconstructing detailed 3D human appearance and geometry, including unobserved back regions, from a single image with real-time rendering and generalization to loose clothing, challenging poses, and in-the-wild images.
Approach
A generate-then-refine pipeline: coarse Gaussians are predicted with a UNet plus an SMPL-X volume branch (learnable tokens on SMPL-X vertices, sparse convolution, transformer fusion); a ControlNet (tile-initialized, on Stable Diffusion 1.5) refines the coarse back-view rendering; a second HGM pass fuses the original front view and refined back view. SMPL-X is iteratively refined using the HGM's si
Key results
- On THuman2.0 novel view synthesis, HGM reaches 23.54 PSNR / 0.938 SSIM / 0.0524 LPIPS, versus 22.10 PSNR for the next best (SIFU).
- On HuMMan, HGM achieves 23.86 PSNR vs. 20.83 for SHERF.
- On CustomHumans 3D reconstruction, HGM reports Chamfer 1.729 and NC 0.834, without 3D supervision, outperforming supervised baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the SMPL-X dual branch for structural human priors (ablation) | HGM · Ablation evaluation (dataset not specified) | PSNR: 21.85 -> 23.54 | Helped |
| Add iterative SMPL-X refinement (ablation) | HGM · Ablation evaluation (dataset not specified) | PSNR: 22.86 -> 23.54 | Helped |
| Add two-view refinement (ablation) | HGM · Ablation evaluation (dataset not specified) | PSNR: 22.95 -> 23.54 | Helped |
| Add diffusion-based back-view refinement (ablation) | HGM · Ablation evaluation (dataset not specified) | PSNR: 23.11 -> 23.54 | Helped |
| Add the SMPL-X dual branch for structural human priors (ablation) | HGM · Ablation evaluation (dataset not specified) | Chamfer: 2.421 -> 2.134 | Helped |
| Add iterative SMPL-X refinement (ablation) | HGM · Ablation evaluation (dataset not specified) | Chamfer: 2.245 -> 2.134 | Helped |
| Add two-view refinement (ablation) | HGM · Ablation evaluation (dataset not specified) | Chamfer: 2.301 -> 2.134 | Helped |
| Add diffusion-based back-view refinement (ablation) | HGM · Ablation evaluation (dataset not specified) | Chamfer: 2.145 -> 2.134 | Helped |
Surprising findings
- Ablation without the SMPL-X dual branch degrades NVS PSNR to 21.85, showing the body prior is critical for side views.
- The paper reports that on THuman2.0 3D reconstruction, SIFU has a lower Chamfer (2.063) than HGM (2.134), so HGM is not best on every geometry metric.
Limitations
- Hands and faces are hard to reconstruct at high quality.
- Mesh extraction from Gaussians is not straightforward and needs extra normal-map optimization.
- Inaccurate initial SMPL-X estimates still affect results; refinement depends on the SMPL-X estimator.
Experimental setup
Datasets: THuman2.0 CustomHumans HuMMan
Benchmarks: THuman2.0 CustomHumans HuMMan
Metrics: PSNR SSIM LPIPS Chamfer Distance P2S Normal Consistency MPJPE KID
Baselines: GTA SiTH LGM SV3D SIFU ECON SHERF MPS-NeRF NHP TeCH
Builds on: 3D Gaussian Splatting Splatter Image ControlNet SMPL-X PIXIE
Closest papers by meaning
- RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images (CVPR 2025 · similarity 0.89)
- HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors (NeurIPS 2024 · similarity 0.88)
- GaussianIP: Identity-Preserving Realistic 3D Human Generation via Human-Centric Diffusion Prior (CVPR 2025 · similarity 0.87)
- HUGS: Human Gaussian Splats (CVPR 2024 · similarity 0.87)
- GauHuman: Articulated Gaussian Splatting from Monocular Human Videos (CVPR 2024 · similarity 0.87)
- No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.