Research map / 3D vision / Face and shape
Gaussian Eigen Models for Human Heads
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Distills per-subject CNN-based 3D Gaussian head avatars into a network-free GEM, an ensemble of PCA eigenbases over Gaussian attributes, driven by a regressor from single images for real-time reenactment.
Problem
High-quality animatable 3D Gaussian head avatars rely on heavy CNN architectures and large checkpoints, which are unsuitable for commodity devices; a lightweight, controllable alternative is needed.
Approach
Represent a personalized head avatar as linear PCA eigenbases over the position, rotation, opacity and scale of 3D Gaussians (color kept global), distilled from a per-subject CNN Gaussian-map model, then refined with photometric loss; a small MLP on EMOCA features regresses the coefficients from a single image.
Key results
- On novel expressions and views, GEM reaches 32.68 PSNR and 0.0675 LPIPS vs. 29.01 PSNR for Animatable Gaussians and 28.31 for Gaussian Avatars.
- A 10-component GEM is 7 MB with 31.81 PSNR at 1282 resolution, roughly 12x smaller than the smallest CNN model and about 70x smaller than Animatable Gaussians.
- GEM renders at 201.70 FPS for cross-reenactment on an A100, versus 16.51 FPS for Animatable Gaussians and 35.77 FPS for the CNN model, with FID 0.4286.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GEM instead of AG [30] | GEM · Novel viewpoint | PSNR: 32.4166 -> 33.5528 | Helped |
| Use GEM instead of AG [30] | GEM · Novel viewpoint | LPIPS: 0.0712 -> 0.0678 | Helped |
| Use GEM instead of AG [30] | GEM · Novel viewpoint | SSIM: 0.9614 -> 0.9662 | Helped |
| Use GEM instead of AG [30] | GEM · Novel viewpoint | L1: 0.0066 -> 0.0061 | Helped |
| Use GEM instead of AG [30] | GEM · Novel expressions and views | PSNR: 29.0114 -> 32.6781 | Helped |
| Use GEM instead of AG [30] | GEM · Novel expressions and views | LPIPS: 0.0812 -> 0.0675 | Helped |
| Use GEM instead of AG [30] | GEM · Novel expressions and views | SSIM: 0.9429 -> 0.9633 | Helped |
| Use GEM instead of AG [30] | GEM · Novel expressions and views | L1: 0.0099 -> 0.0069 | Helped |
| Use GEM instead of Ours Net | GEM · Cross-reenactment | Efeatcos: 0.9440 -> 0.9381 | Hurt |
| Use GEM instead of Ours Net | GEM · Cross-reenactment | FPS: 35.77 -> 201.70 | Helped |
| Use GEM instead of Ours Net | GEM · Cross-reenactment | FID: 0.3685 -> 0.4286 | Hurt |
| Increase the basis size from 10 to 30 components (ablation) | GEM · Novel view, 128² Gaussian maps | PSNR: 31.81 -> 34.20 | Helped |
Surprising findings
- The linear PCA model (GEM) outperforms the CNN-based appearance model it was distilled from on novel views and expressions (PSNR 32.68 vs 29.25 on novel expressions).
Limitations
- PCA bases have a global extent, so local expression changes cannot be controlled independently and generalization beyond training expressions is limited.
- Side-view generalization yields unstable expressions.
- Each new subject requires a new representation learned from multi-view data; no cross-subject model.
Experimental setup
Datasets: NeRSemble
Benchmarks: NeRSemble
Metrics: PSNR LPIPS SSIM L1 EmoNet feature cosine similarity EmoNet feature L1 FID FPS model size MB
Baselines: Gaussian Avatars (GA) Animatable Gaussians (AG) INSTA StyleAvatar-style CNN (Ours Net)
Builds on: Animatable Gaussians 3D Gaussian Splatting EMOCA 3D morphable model Gaussian Avatars
Closest papers by meaning
- 3D Gaussian Head Avatars with Expressive Dynamic Appearances by Compact Tensorial Representations (CVPR 2025 · similarity 0.91)
- Generalizable and Animatable Gaussian Head Avatar (NeurIPS 2024 · similarity 0.91)
- Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians (CVPR 2024 · similarity 0.90)
- GPAvatar: High-fidelity Head Avatars by Learning Efficient Gaussian Projections (CVPR 2025 · similarity 0.90)
- Gaussian Head & Shoulders: High Fidelity Neural Upper Body Avatars with Anchor Gaussian Guided Texture Warping (ICLR 2025 · similarity 0.90)
- RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatars (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.