Research map / 3D vision / Face and shape
FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FATE, an animatable 3D Gaussian head avatar method from monocular video with UV-space sampling-based densification, neural baking into editable attribute maps, and a SphereHead-based completion framework for 360° rendering.
Problem
Monocular video head avatar reconstruction is incomplete (rear/side head not recovered) and 3D Gaussian representations are inefficient (redundant points from threshold densification) and not editable in UV texture space.
Approach
Replaces gradient-threshold densification with multinomial sampling of Gaussians by gradient importance and resamples new positions uniformly within the bound triangle; uses a U-Net BakeNet as a pre-filter during a two-stage training to bake Gaussian attributes into smooth UV attribute maps; and completes the rear head by PTI-inverting SphereHead generated multi-view images.
Key results
- Overall PSNR 28.37, SSIM 0.9439, LPIPS 0.0586 vs best baseline MonoGaussianAvatar 28.07 / 0.9405 / 0.0787 across the 20-subject evaluation.
- Uses only 49k±6k Gaussians on INSTA versus 72k±33k for GaussianAvatars and 558k±188k for SplattingAvatar, with more stable counts across datasets.
- Ablation on yufeng subject shows removing sampling-based densification drops PSNR from 29.36 to 29.13 and LPIPS rises from 0.0694 to 0.0740.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FATE instead of MonoGaussianAvatar | monocular head avatar reconstruction · Overall | PSNR: 28.07 -> 28.37 | Helped |
| use FATE instead of MonoGaussianAvatar | monocular head avatar reconstruction · Overall | SSIM: 0.9405 -> 0.9439 | Helped |
| use FATE instead of FlashAvatar | monocular head avatar reconstruction · Overall | LPIPS: 0.0603 -> 0.0586 | Helped |
| add sampling-based densification (ablation) | FATE · yufeng case | LPIPS: 0.0740 -> 0.0694 | Helped |
| add learnable expression and pose blendshapes (ablation) | FATE · yufeng case | PSNR: 24.78 -> 29.36 | Helped |
| use two-stage baking instead of one-stage baking (ablation) | FATE · yufeng case | PSNR: 27.42 -> 27.78 | Helped |
| use BakeNet encoding rather than only its decoder (ablation) | FATE neural baking · yufeng case | PSNR: 25.56 -> 27.78 | Helped |
Surprising findings
- Directly optimizing UV texture maps of Gaussian attributes yields visibly holed, discontinuous textures, motivating the CNN pre-filter in neural baking.
- Baked texture version scores slightly below the non-baked model (overall PSNR 28.23 vs 28.37), a trade-off for editability.
Limitations
- Assumes consistent and uniform lighting, reducing robustness in real-world scenarios.
- Completion depends on SphereHead's training distribution and may cause identity change for personalized head shapes.
- Fixed-size texture maps from neural baking may fail in some cases.
Experimental setup
Datasets: INSTA PointAvatar NerFace EmoTalk3D FFHQ-style SphereHead training data
Benchmarks: INSTA PointAvatar NerFace EmoTalk3D
Metrics: PSNR SSIM LPIPS number of Gaussians
Baselines: FlashAvatar SplattingAvatar MonoGaussianAvatar GaussianAvatars
Builds on: 3D Gaussian Splatting FLAME SphereHead Pivotal Tuning Inversion PhysGaussian
Closest papers by meaning
- GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion (CVPR 2025 · similarity 0.89)
- Generalizable and Animatable Gaussian Head Avatar (NeurIPS 2024 · similarity 0.89)
- GPAvatar: High-fidelity Head Avatars by Learning Efficient Gaussian Projections (CVPR 2025 · similarity 0.89)
- Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians (CVPR 2024 · similarity 0.89)
- HRAvatar: High-Quality and Relightable Gaussian Head Avatar (CVPR 2025 · similarity 0.89)
- RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatars (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.