Research map / 3D vision / Gaussian splatting
SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents SplattingAvatar, a hybrid avatar representation that embeds 3D Gaussians on a triangle mesh via trainable barycentric embeddings and lifted optimization, enabling real-time rendering from monocular video.
Problem
Photorealistic, animatable human avatars from monocular video are typically NeRF- or MLP-deformation-based, which is slow and ambiguous to render; mesh-based avatars lack high-frequency detail and real-time mobile performance is hard to achieve.
Approach
Gaussians are embedded on a triangle mesh as Phong surfaces (triangle index, barycentric coordinates, normal displacement), with pose-dependent rotation and scaling derived from mesh deformation; the embeddings are optimized jointly with Gaussian parameters via lifted optimization that lets points walk across triangle boundaries.
Key results
- On head avatars, Ours+NHA reaches 28.86 PSNR / 0.931 SSIM / 0.060 LPIPS vs. PointAvatar 27.84 / 0.913 / 0.067.
- On PeopleSnapshot, Ours achieves the highest PSNR on all four subjects (e.g., 33.01 on male-3-casual vs. InstantAvatar 30.91).
- Renders over 300 FPS on an NVIDIA RTX 3090 and 30 FPS on an iPhone 13 in Unity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use mesh-embedded Gaussian Splatting instead of the compared methods | Head avatars · Head avatars | PSNR: 27.84 -> 28.86 | Helped |
| Use mesh-embedded Gaussian Splatting instead of the compared methods | Head avatars · Head avatars | LPIPS: 0.067 -> 0.060 | Helped |
| Use mesh-embedded Gaussian Splatting instead of InstantAvatar | PeopleSnapshot · PeopleSnapshot male-3-casual | PSNR: 30.91 -> 33.01 | Helped |
| Use mesh-embedded Gaussian Splatting instead of InstantAvatar | PeopleSnapshot · PeopleSnapshot male-3-casual | SSIM: 0.977 -> 0.982 | Helped |
| Use mesh-embedded Gaussian Splatting instead of InstantAvatar | PeopleSnapshot · PeopleSnapshot male-3-casual | LPIPS: 0.022 -> 0.020 | Hurt |
| Remove trainable mesh embedding and use fixed embedding with a trainable local shift (ablation) | SplattingAvatar · Novel-pose rendering | rendering quality | Hurt |
| Remove the scaling regularization term (ablation) | SplattingAvatar · Novel-pose rendering | rendering quality | Hurt |
Surprising findings
- Driving mesh should emphasize motion rather than exact geometry: the FLAME mesh (Ours+FLAME) gave better visual quality in some inner regions than the more detailed NHA mesh despite lower average metrics for that variant.
Limitations
- Quality depends on the motion representation of the driving mesh; FLAME and SMPL lack separate motion for hair and clothes.
- Off-surface geometry such as hair and glasses is not fully represented by the mesh.
- Limited pose variation in monocular training videos causes artifacts (e.g., under the shoulder).
Experimental setup
Datasets: NHA NerFace INSTA PointAvatar PeopleSnapshot
Benchmarks: NHA/NerFace/INSTA/PointAvatar combined head dataset PeopleSnapshot
Metrics: PSNR SSIM LPIPS FPS
Baselines: NHA INSTA PointAvatar Anim-NeRF InstantAvatar
Builds on: 3D Gaussian Splatting lifted optimization Phong surface INSTA InstantAvatar
Closest papers by meaning
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.93)
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (CVPR 2024 · similarity 0.91)
- ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Rendering (CVPR 2024 · similarity 0.90)
- GauHuman: Articulated Gaussian Splatting from Monocular Human Videos (CVPR 2024 · similarity 0.90)
- Creating Your Editable 3D Photorealistic Avatar with Tetrahedron-constrained Gaussian Splatting (CVPR 2025 · similarity 0.90)
- MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.