Research map / 3D vision / Gaussian splatting
HUGS: Human Gaussian Splats
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces HUGS, an animatable human avatar plus static scene represented jointly with 3D Gaussian Splatting, trained from a short monocular video in about 30 minutes and rendered at 60 FPS.
Problem
Neural rendering methods such as 3DGS target static scenes and do not handle freely moving, animatable humans embedded in scenes; NeRF-based human avatars are slow to train and render.
Approach
Represent the human as 3D Gaussians initialized from the SMPL mesh, placed in a canonical space and decoded by a triplane plus three MLPs (appearance, geometry, LBS deformation), with LBS weights jointly optimized and regularized toward SMPL, and combined with static scene Gaussians so the human can be transferred to other scenes.
Key results
- On NeuMan full-frame test images HUGS reaches PSNR 25.94 on Seattle and LPIPS 0.09 on Lab, leading on nearly all sequences (except PSNR on Bike).
- On ZJU-MoCap HUGS achieves the best SSIM and LPIPS across all six subjects in the reported comparison, e.g. PSNR 34.11 on subject 386.
- Training takes about 30 minutes (96x faster than Vid2Avatar and 336x faster than NeuMan) with rendering at 60 FPS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HUGS instead of NeuMan | Human-scene reconstruction · NeuMan dataset: Seattle, Citron, Parking, Jogging and Lab test sequences | PSNR: 23.99 -> 25.94; 24.63 -> 25.54; 25.43 -> 26.86; 22.70 -> 23.75; 24.96 -> 26.00 | Helped |
| Use HUGS instead of NeuMan | Human-scene reconstruction · NeuMan dataset: Bike test sequence | PSNR: 25.55 -> 25.46 | Hurt |
| Use HUGS instead of NeuMan | Human reconstruction · ZJU-MoCap subject 386 | PSNR: 33.20 -> 34.11 | Helped |
| Use HUGS instead of the strongest listed baseline | Human reconstruction · ZJU-MoCap subjects 377, 392 and 394 | PSNR: 31.36 -> 30.80; 32.03 -> 31.36; 31.46 -> 30.54 | Hurt |
| Add learned LBS weights for Gaussian animation (ablation) | HUGS · NeuMan human-only bounding boxes, Seattle | PSNR: 18.47 -> 19.06 | Helped |
| Add Gaussian densification (ablation) | HUGS · NeuMan human-only bounding boxes, Seattle | PSNR: 18.91 -> 19.06 | Helped |
| Add human-region loss Lh (ablation) | HUGS · NeuMan human-only bounding boxes, Seattle | PSNR: 18.87 -> 19.06 | Helped |
| Use the triplane and MLP instead of directly optimizing 3DGS parameters (ablation) | HUGS · NeuMan human-only bounding boxes, Seattle | PSNR: 18.70 -> 19.06 | Helped |
| Add learned LBS weights for Gaussian animation (ablation) | HUGS · NeuMan human-only bounding boxes, Citron | PSNR: 19.00 -> 19.16 | Helped |
| Add Gaussian densification (ablation) | HUGS · NeuMan human-only bounding boxes, Citron | PSNR: 17.18 -> 19.16 | Helped |
| Add human-region loss Lh (ablation) | HUGS · NeuMan human-only bounding boxes, Citron | PSNR: 17.31 -> 19.16 | Helped |
| Use the triplane and MLP instead of directly optimizing 3DGS parameters (ablation) | HUGS · NeuMan human-only bounding boxes, Citron | PSNR: 17.12 -> 19.16 | Helped |
Surprising findings
- Rendering speed is decoupled from the MLPs at test time because the Gaussians are explicitly animated via LBS weights.
Limitations
- Limited by SMPL and linear blend skinning, which do not capture loose clothing such as dresses.
- In-the-wild training videos do not cover the full pose space of the human body.
- Does not model environment lighting, so compositing into new scenes may mismatch illumination.
Experimental setup
Datasets: NeuMan dataset ZJU-MoCap dataset
Benchmarks: NeuMan dataset ZJU-MoCap dataset
Metrics: PSNR SSIM LPIPS training time rendering FPS
Baselines: NeRF-T HyperNeRF NeuMan Vid2Avatar NeuralBody HumanNeRF InstantNVR MonoHuman
Builds on: 3D Gaussian Splatting SMPL NeuMan Vid2Avatar
Closest papers by meaning
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.95)
- GauHuman: Articulated Gaussian Splatting from Monocular Human Videos (CVPR 2024 · similarity 0.92)
- ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Rendering (CVPR 2024 · similarity 0.90)
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (CVPR 2024 · similarity 0.90)
- Gaussian Head & Shoulders: High Fidelity Neural Upper Body Avatars with Anchor Gaussian Guided Texture Warping (ICLR 2025 · similarity 0.90)
- SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.