Research map / 3D vision / Face and shape
Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Gaussian human avatar representation with spatially distributed pose-driven MLPs whose outputs are interpolated as coefficients over learned Gaussian offset bases, plus control-point position interpolation, enabling high-fidelity avatars rendered at 166 FPS.
Problem
Gaussian human avatars either use a single MLP that cannot capture high-frequency pose-dependent appearance, or use heavy CNNs (e.g., StyleUNet) that yield high fidelity but non-real-time rendering.
Approach
Spatially distributed MLPs on anchor points take only pose as input; their outputs are distance-interpolated as coefficients over per-Gaussian learnable offset bases, so smoothly varying coefficients still yield sharp, distinct property offsets. Gaussian position offsets are interpolated from control points on the template surface to keep Gaussians on a surface layer.
Key results
- Achieves PSNR 32.75, SSIM 0.9868, LPIPS 0.0226, FID 10.12 on training poses, beating AnimatableGaussians (31.30 PSNR).
- Renders at 166 FPS with 200K Gaussians on an NVIDIA 3090 versus 10 FPS for AnimatableGaussians.
- Training takes 17.5 hours versus 100 hours for AnimatableGaussians under 800K iterations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed method instead of AnimatableGaussians | Gaussian human avatar reconstruction · Training poses | PSNR: 31.2992 -> 32.7456 | Helped |
| Use the proposed method instead of AnimatableGaussians | Gaussian human avatar reconstruction · Novel poses | PSNR: 28.1106 -> 28.3263 | Helped |
| Use the proposed method instead of AnimatableGaussians | Gaussian human avatar rendering · Novel poses | FPS: 10 -> 166 | Helped |
| Use a single MLP instead of spatially distributed MLPs (ablation) | Ours · THuman4.0 subject02 training poses | PSNR: 32.7456 -> 30.1941 | Hurt |
| Add Gaussian offset basis rather than directly interpolating property offsets (ablation) | Ours · THuman4.0 subject02 training poses | PSNR: 31.8444 -> 32.7456 | Helped |
| Add Gaussian offset basis rather than directly interpolating property offsets (ablation) | Ours · THuman4.0 subject02 training poses | PSNR: 32.3912 -> 32.7456 | Helped |
| Combine offset basis instead of predicting Gaussian properties directly (ablation) | Ours · THuman4.0 subject02 training poses | PSNR: 31.9340 -> 32.7456 | Helped |
| Add control-point interpolation for Gaussian position offsets (ablation) | Ours · THuman4.0 subject02 training poses | PSNR: 32.6780 -> 32.7456 | Helped |
| Use control-point interpolation rather than an MLP for position offsets (ablation) | Ours · THuman4.0 subject02 training poses | PSNR: 31.8514 -> 32.7456 | Helped |
| Increase spatially distributed MLP count from 50 to 300 (ablation) | Ours · THuman4.0 subject02 training poses | PSNR: 32.6625 -> 32.7456 | Helped |
| Increase spatially distributed MLP count from 300 to 800 (ablation) | Ours · THuman4.0 subject02 training poses | FPS: 166 -> 149 | Hurt |
| Increase Gaussian offset basis count from 5 to 15 (ablation) | Ours · THuman4.0 subject02 training poses | PSNR: 32.3523 -> 32.7456 | Helped |
Limitations
- Appearance is conditioned on pose and cannot model complex cloth dynamics such as long skirts.
- Relies on multi-view capture, pose estimation, and template mesh extraction, making the pipeline heavy.
- Requires multi-view videos rather than monocular input.
Experimental setup
Datasets: AvatarRex THuman4.0 ActorsHQ
Benchmarks: AvatarRex THuman4.0 ActorsHQ
Metrics: PSNR SSIM LPIPS FID FPS training time
Baselines: 3DGS-Avatar MeshAvatar AnimatableGaussians DEGAS PoseVocab
Builds on: 3D Gaussian Splatting AnimatableGaussians SMPL-X
Closest papers by meaning
- Gaussian Head & Shoulders: High Fidelity Neural Upper Body Avatars with Anchor Gaussian Guided Texture Warping (ICLR 2025 · similarity 0.91)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.90)
- GASP: Gaussian Avatars with Synthetic Priors (CVPR 2025 · similarity 0.90)
- GPAvatar: High-fidelity Head Avatars by Learning Efficient Gaussian Projections (CVPR 2025 · similarity 0.90)
- Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians (CVPR 2024 · similarity 0.90)
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.