Research map / 3D vision / Face and shape
Gaussian Head & Shoulders: High Fidelity Neural Upper Body Avatars with Anchor Gaussian Guided Texture Warping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a monocular upper-body (head, chest, shoulders) avatar by combining FLAME-driven 3D Gaussians for the head with a neural texture warped through sparse anchor Gaussians, plus an MLP-free accelerated renderer reaching about 130 FPS.
Problem
Reconstructing realistic, controllable upper-body avatars (head plus clothed chest and shoulders) from casual monocular videos, where plain Gaussian splatting produces blur and floaters on high-frequency clothing under novel poses.
Approach
Model the clothed upper body as a learned neural texture rather than Gaussians, and use a sparse set of FLAME-LBS-driven anchor Gaussians to supervise a coordinate-MLP warp from image pixels to texture space; then distill the MLPs into a static texture with a homography at inference for fast rendering.
Key results
- Full method achieves best PSNR/SSIM/LPIPS on most of the 7 self-reenactment subjects (e.g., subject 001 PSNR 25.95 vs 23.26 for GS*), with the paper reporting clear gains over INSTA, PointAvatar, SplattingAvatar and GS*.
- The no-MLP accelerated version renders at about 130 FPS versus about 70 FPS for the full model and uses about 40k-59k Gaussians on subjects 003, 004, 005, 007 versus up to 318k for GS*.
- On body-only comparison with GSAvatar, the full method reaches PSNR 26.82 vs 17.08 on subject 004 and 23.87 vs 16.64 on subject 007.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use anchor Gaussians to constrain texture warping (ablation) | Gaussian Head & Shoulders · Self-reenactment 002, 005, 007 | PSNR: 24.96 -> 31.98; 22.91 -> 24.48; 19.30 -> 23.26 | Mixed |
| Use anchor Gaussians to constrain texture warping (ablation) | Gaussian Head & Shoulders · Self-reenactment 002, 005, 007 | LPIPS: .088 -> .042; .117 -> .074; .134 -> .074 | Helped |
| Remove the warp loss (ablation) | Gaussian Head & Shoulders · Self-reenactment 002, 005, 007 | PSNR: 31.98 -> 32.86; 24.48 -> 24.19; 23.26 -> 22.74 | Mixed |
| Remove MLPs at inference and use cached texture with homography (ablation) | Gaussian Head & Shoulders · Self-reenactment | rendering speed: around 70 FPS to 130 FPS | Helped |
| Remove MLPs at inference (ablation) | Gaussian Head & Shoulders · Self-reenactment | PSNR: 25.95 -> 24.48; 31.98 -> 31.44; 31.26 -> 28.85; 24.68 -> 24.61; 24.48 -> 22.19; 30.97 -> 28.71; 23.26 -> 21.49 | Mixed |
| Use the full method rather than GS* | Gaussian Head & Shoulders · Self-reenactment 001, 003–007 | PSNR: 23.26 -> 25.95; 29.85 -> 31.26; 24.18 -> 24.68; 22.80 -> 24.48; 29.56 -> 30.97; 22.31 -> 23.26 | Helped |
| Use the full method rather than GSAvatar | Gaussian Head & Shoulders · Body-only comparison 004, 007 | PSNR: 17.08 -> 26.82; 16.64 -> 23.87 | Helped |
| Apply test-time Euclidean transformation to align body texture with head Gaussians (ablation) | Gaussian Head & Shoulders · Cross-reenactment | body-texture alignment | Helped |
| Apply warp loss when combining neural warping with Euclidean transformation (ablation) | Gaussian Head & Shoulders · Cross-reenactment | background rendering quality | Helped |
| Remove texture cleaning from the No MLP inference pipeline (ablation) | Gaussian Head & Shoulders (No MLP) · Novel-pose rendering | visual artifacts | Hurt |
Surprising findings
- The no-MLP version often has lower PSNR than baselines but higher LPIPS, which the authors attribute to PSNR favoring blurry results.
- The no-MLP version is faster than plain GS* for the subject with high-frequency cloth (005: 132 vs 96 FPS), even though GS* is faster on other subjects.
Limitations
- Cannot handle extreme body rotation causing self-occlusion because the neural texture warping does not model occlusion.
- The no-MLP homography version cannot model non-rigid body deformation and may misalign under extreme head rotation.
- The full model's texture-warping MLP can overfit to cloth-deformation noise, causing unnatural deformation for subjects 004 and 007.
Experimental setup
Datasets: PointAvatar mobile sequence Head2Head dataset sequences self-captured mobile phone sequences
Benchmarks: PointAvatar mobile sequence Head2Head internet sequences self-captured phone sequences
Metrics: PSNR SSIM LPIPS FPS number of Gaussians
Baselines: INSTA PointAvatar SplattingAvatar GS* GSAvatar
Builds on: 3D Gaussian Splatting FLAME PointAvatar GaussianAvatars IMAvatar
Closest papers by meaning
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.92)
- Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs (CVPR 2025 · similarity 0.91)
- Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians (CVPR 2024 · similarity 0.90)
- GPAvatar: High-fidelity Head Avatars by Learning Efficient Gaussian Projections (CVPR 2025 · similarity 0.90)
- Gaussian Eigen Models for Human Heads (CVPR 2025 · similarity 0.90)
- HUGS: Human Gaussian Splats (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.