Research map / 3D vision / Gaussian splatting
HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HumanSplat, a feed-forward network that predicts 3D Gaussian Splatting properties of a human from one image by combining a fine-tuned SV3D multi-view diffusion model with a latent reconstruction Transformer that uses SMPL priors and semantics-guided losses.
Problem
Single-image 3D human reconstruction typically needs dense views or slow per-instance optimization (SDS, ~hours), and existing generalizable methods either ignore human priors or need multi-view inputs.
Approach
Reconstruct pixel-aligned Gaussians from the latent tokens of a fine-tuned multi-view video diffusion model, with projection-aware windowed cross-attention to SMPL geometric tokens (robust to imperfect SMPL) and a hierarchical loss weighted by body-part semantics (head, hands emphasized).
Key results
- On Twindom, HumanSplat reaches 23.346 PSNR / 0.125 LPIPS versus TeCH's 21.192 / 0.188, while running in 9.3s versus TeCH's 4.5 hours.
- On THuman2.0, HumanSplat achieves the best LPIPS (0.055) among methods, with PSNR 24.033 slightly below TeCH's 25.211.
- Removing the SMPL human prior drops 2K2K PSNR from 24.293 to 22.635.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune SV3D on human datasets | SV3D novel-view synthesizer · Human novel-view generation | matching correspondences: 723.25 -> 930.13 | Helped |
| use latent space instead of pixel space (ablation) | HumanSplat · 2K2K evaluation | PSNR: 23.125 -> 24.293 | Helped |
| use vanilla VAE decoding instead of Temporal VAE decoding (ablation) | HumanSplat · 2K2K evaluation | PSNR: 24.133 -> 24.293 | Helped |
| add the human structure prior (ablation) | HumanSplat · 2K2K evaluation | PSNR: 22.635 -> 24.293 | Helped |
| use ground-truth SMPL initialization instead of the standard initialization (ablation) | HumanSplat · 2K2K evaluation | PSNR: 24.293 -> 24.633 | Helped |
| use projection-aware attention with K_win=2 instead of projection-only attention (ablation) | HumanSplat · 2K2K evaluation | PSNR: 23.333 -> 24.293 | Helped |
| set projection window size to K_win=3 instead of K_win=2 (ablation) | HumanSplat · 2K2K evaluation | PSNR: 24.293 -> 24.383 | Helped |
| set projection window size to K_win=4 instead of K_win=2 (ablation) | HumanSplat · 2K2K evaluation | PSNR: 24.293 -> 24.013 | Hurt |
| add reweighting loss (ablation) | HumanSplat · 2K2K evaluation | PSNR: -0.141 | Hurt |
| add reweighting loss (ablation) | HumanSplat · 2K2K evaluation | SSIM: -0.015 | Hurt |
| add reweighting loss (ablation) | HumanSplat · 2K2K evaluation | LPIPS: +0.006 | Hurt |
| use HumanSplat instead of SIFU | single-image human reconstruction · THuman2.0 | PSNR: +1.92 | Helped |
Surprising findings
- TeCH achieves higher PSNR on THuman2.0 (25.211 vs 24.033) even though HumanSplat has better LPIPS, indicating PSNR and perceptual quality disagree.
Limitations
- May struggle with intricate garments and accessories.
- Animation of reconstructed models requires post-processing; no canonical space.
- Inference still needs about 9s for diffusion-based novel view generation.
Experimental setup
Datasets: THuman2.0 2K2K Twindom
Benchmarks: THuman2.0 Twindom 2K2K
Metrics: PSNR SSIM LPIPS
Baselines: PIFu LGM GTA SIFU Magic123 HumanSGD TeCH
Builds on: SV3D 3D Gaussian Splatting LGM SMPL PIXIE
Closest papers by meaning
- RoGSplat: Learning Robust Generalizable Human Gaussian Splatting from Sparse Multi-View Images (CVPR 2025 · similarity 0.90)
- HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction (ICLR 2025 · similarity 0.90)
- Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpses (CVPR 2024 · similarity 0.90)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.90)
- pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction (CVPR 2024 · similarity 0.89)
- HUGS: Human Gaussian Splats (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.