Research map / 3D vision / Gaussian splatting
Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Creates animatable human avatars by learning a character-specific template and predicting pose-dependent front/back Gaussian maps with a StyleUNet CNN, plus a PCA-based pose projection for novel poses.
Problem
Modeling high-fidelity, pose-dependent garment details and generalizing to novel poses for animatable human avatars learned from multi-view RGB videos, where pure MLP-based NeRFs struggle with high-frequency detail.
Approach
Parameterize 3D Gaussians anchored on a learned character-specific template onto front and back orthographic canonical Gaussian maps, so a StyleGAN-based 2D CNN (StyleUNet) can regress pose-dependent Gaussian attributes; a PCA pose projection keeps driving poses within the training distribution.
Key results
- On THuman4.0 subject00, the method reaches PSNR 28.07, SSIM 0.9739, LPIPS 0.0515, FID 29.48, beating PoseVocab (PSNR 26.38, LPIPS 0.0592).
- Against AvatarReX on its dataset, PSNR 30.61 vs 23.25 and LPIPS 0.0290 vs 0.0646 for training-pose reconstruction.
- Renders at 25 FPS in PyTorch on an RTX 3090 at 1024x1024, versus 0.003-0.20 FPS for compared NeRF-based methods (TAVA, ARAH, SLRF, PoseVocab, AvatarReX).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Animatable Gaussians instead of PoseVocab | THuman4.0 subject00 · THuman4.0 subject00 testing chunk | PSNR: 26.3784 -> 28.0714 | Helped |
| Use Animatable Gaussians instead of AvatarReX | AvatarReX dataset · AvatarReX dataset training pose reconstruction | PSNR: 23.2475 -> 30.6143 | Helped |
| Replace the character-specific template with SMPL-X (ablation) | Animatable Gaussians · Novel pose synthesis | visual avatar quality | Hurt |
| Replace StyleUNet with a coordinate-based MLP (ablation) | Animatable Gaussians · Training pose reconstruction | visual detail | Hurt |
| Replace StyleUNet with a standard U-Net (ablation) | Animatable Gaussians · Training pose reconstruction | visual detail | Hurt |
| Remove pose projection and directly input the novel position map (ablation) | Animatable Gaussians · Novel pose synthesis | visual avatar quality | Hurt |
Limitations
- Entangles body and clothing modeling, so clothes cannot be changed independently (e.g., for virtual try-on).
- Requires multi-view input to reconstruct the parametric template, limiting application to loose clothes from monocular video.
Experimental setup
Datasets: THuman4.0 ActorsHQ AvatarReX dataset
Benchmarks: THuman4.0 ActorsHQ AvatarReX dataset
Metrics: PSNR SSIM LPIPS FID framerate (FPS)
Baselines: PoseVocab SLRF ARAH TAVA AvatarReX U-Net backbone MLP backbone
Builds on: 3D Gaussian splatting StyleGAN SDF-based volume rendering (NeuS) SMPL-X PointAvatar
Closest papers by meaning
- GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians (CVPR 2024 · similarity 0.91)
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (CVPR 2024 · similarity 0.91)
- ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Rendering (CVPR 2024 · similarity 0.90)
- GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning (CVPR 2024 · similarity 0.90)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.90)
- Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.