Research map / 3D vision / Gaussian splatting
GauHuman: Articulated Gaussian Splatting from Monocular Human Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GauHuman, an articulated 3D Gaussian Splatting model that reconstructs animatable humans from monocular videos in 1-2 minutes of training with real-time rendering (up to 189 FPS).
Problem
Subject-specific NeRF-based human avatars from monocular video need hours of training and seconds per frame at rendering, which limits practical use.
Approach
Represents the human as 3D Gaussians in canonical space, deforms them to posed space with LBS whose weights are SMPL weights plus MLP offsets and whose pose is refined by an MLP; initializes Gaussians from SMPL vertices, controls densification with KL-divergence-guided split/clone, adds a merge operation and SMPL-based pruning.
Key results
- On ZJU_MoCap, GauHuman reaches 31.34 PSNR in about 1 minute of training at 189 FPS, versus InstantNVR at 31.01 PSNR with 5 minutes and 2.20 FPS.
- On MonoCap, GauHuman reaches 33.45 PSNR in 2 minutes at 154 FPS, the best PSNR among the compared methods in Table 1.
- Ablation: KL-based split/clone converges about 60x faster than the original 3DGS split/clone (1h vs 55s training with 100k vs 13k Gaussians).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add pose refinement (ablation) | GauHuman · ZJU_MoCap (386 sequence, masked area) | PSNR: 27.94 -> 28.08 | Helped |
| add LBS weight field (ablation) | GauHuman · ZJU_MoCap (386 sequence, masked area) | PSNR: 27.87 -> 28.08 | Helped |
| use articulated rather than random initialization at #Gau 6890 (ablation) | GauHuman · ZJU_MoCap (386 sequence, masked area) | PSNR: 27.98 -> 28.08 | Helped |
| use articulated rather than random initialization (ablation) | GauHuman · ZJU_MoCap (386 sequence, masked area) | Train: 5m -> 55s | Helped |
| add KL-based split/clone guidance (ablation) | GauHuman · ZJU_MoCap (386 sequence, masked area) | Train: 1h -> 55s | Helped |
| add KL-based merge operation (ablation) | GauHuman · ZJU_MoCap (386 sequence, masked area) | PSNR: 28.08 -> 28.08 | No clear effect |
| add articulated pruning (ablation) | GauHuman · ZJU_MoCap (386 sequence, masked area) | PSNR: 28.08 -> 28.08 | No clear effect |
| use #Gau 30k rather than #Gau 6890 for random initialization (ablation) | GauHuman · ZJU_MoCap (386 sequence, masked area) | Train: 5m -> 30m | Hurt |
Surprising findings
- Random initialization with 30k Gaussians only matches the articulated SMPL initialization's PSNR, yet takes about 30 minutes versus 55 seconds.
Limitations
- Cannot extract 3D meshes from the Gaussian representation.
- Struggles with loose or complex clothing from monocular video.
- Relies on known camera parameters, masks and SMPL parameters.
Experimental setup
Datasets: ZJU_MoCap MonoCap
Benchmarks: ZJU_MoCap MonoCap
Metrics: PSNR SSIM LPIPS training time FPS
Baselines: PixelNeRF NHP Neural Body Animatable NeRF Anim-NeRF Neural Actor HumanNeRF DVA InstantNVR InstantAvatar
Builds on: 3D Gaussian Splatting HumanNeRF linear blend skinning (SMPL) InstantAvatar
Closest papers by meaning
- HUGS: Human Gaussian Splats (CVPR 2024 · similarity 0.92)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.92)
- SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting (CVPR 2024 · similarity 0.90)
- Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives (CVPR 2025 · similarity 0.89)
- Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting (ICLR 2025 · similarity 0.89)
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.