Research map / 3D vision / Face and shape
SurFhead: Affine Rig Blending for Geometrically Accurate 2D Gaussian Surfel Head Avatars
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a riggable head avatar from RGB videos by binding 2D Gaussian surfels to FLAME triangles with affine Jacobian deformation, Jacobian Blend Skinning, and eyeball specularity regularization.
Problem
Head avatars built from Gaussian primitives capture geometry poorly and fail to render extreme unseen poses because similarity transforms cannot represent stretch and shear of the parametric mesh; normals are also mishandled under deformation.
Approach
Replaces the similarity transform of mesh-bound Gaussians with the triangle deformation gradient J (affine), transforms normals by J^{-T}, and blends adjacent triangles' Jacobians via polar decomposition (log-space rotation blending, linear-space stretch blending), plus ASG-based specular modeling and opacity regularization for eyeballs.
Key results
- On FaceTalk synthetic data, SurFhead reaches PSNR 40.15, SSIM 0.992, LPIPS 0.020, NCS 0.983, beating IMAvatar, PointAvatar and FLARE.
- On NeRSemble self-reenactment, SurFhead achieves NCS 0.832 versus 0.716 for GaussianAvatars, with PSNR 23.53 vs 23.43.
- On NeRSemble novel-view synthesis, PSNR is 30.07 vs GaussianAvatars' 30.29, while NCS improves to 0.896 vs 0.797.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace similarity-transform rigging with affine Jacobian deformation | SurFhead · FaceTalk | NCS: 0.943 -> 0.983 | Helped |
| use SurFhead instead of GaussianAvatars | NeRSemble novel-view synthesis · NeRSemble novel-view synthesis | NCS: 0.797 -> 0.896 | Helped |
| use SurFhead instead of GaussianAvatars | NeRSemble novel-view synthesis · NeRSemble novel-view synthesis | PSNR: 30.29 -> 30.07 | Hurt |
| add LPIPS training (ablation) | SurFhead · NeRSemble novel-view synthesis | LPIPS: 0.079 -> 0.062 | Helped |
| add LPIPS training (ablation) | SurFhead · NeRSemble self-reenactment | PSNR: 23.53 -> 23.78 | Helped |
| add eyeball treatment (ablation) | Vanilla (2DGS + GaussianAvatars) · NeRSemble ablation study | NCS: 0.803 -> 0.809 | Helped |
| add Jacobian deformation (ablation) | + eyeballs · NeRSemble ablation study | NCS: 0.809 -> 0.812 | Helped |
| add Jacobian Blend Skinning (JBS) (ablation) | + Jacobian · NeRSemble ablation study | NCS: 0.812 -> 0.845 | Helped |
| remove ASGs (ablation) | SurFhead · NeRSemble ablation study | NCS: 0.845 -> 0.846 | No clear effect |
| add ASGs (ablation) | SurFhead · NeRSemble ablation study | LPIPS: 0.089 -> 0.082 | Helped |
| remove eyeball treatment (ablation) | SurFhead · NeRSemble ablation study | NCS: 0.845 -> 0.820 | Hurt |
| add JBS (ablation) | SurFhead with Jacobian · NeRSemble rendering | Rendering speed (FPS): 107.59 -> 92.62 | Hurt |
Surprising findings
- GaussianAvatars and GHA achieve higher PSNR than SurFhead on NeRSemble, which the authors attribute to 3DGS representation advantages over 2D surfels.
- Replacing torch.svd with RoMa's Procrustes routine gives roughly 4-5 FPS speedup in rendering.
Limitations
- Expression space bounded by the FLAME 3DMFM; cannot capture wrinkles or exaggerated expressions outside its span.
- Lack of hair and tongue in the 3DMFM leads to blurry hair appearance.
- Polar decomposition via SVD adds rendering overhead.
Experimental setup
Datasets: NeRSemble FaceTalk
Benchmarks: FaceTalk NeRSemble
Metrics: PSNR SSIM LPIPS NCS
Baselines: IMAvatar FLARE PointAvatar SplattingAvatar GaussianAvatars GHA
Builds on: 2DGS GaussianAvatars FLAME 3DGS Anisotropic Spherical Gaussians
Closest papers by meaning
- Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians (CVPR 2024 · similarity 0.88)
- MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Views (CVPR 2025 · similarity 0.87)
- MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing (CVPR 2025 · similarity 0.87)
- RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatars (CVPR 2025 · similarity 0.87)
- SurfelSplat: Learning Efficient and Generalizable Gaussian Surfel Representations for Sparse-View Surface Reconstruction (NeurIPS 2025 · similarity 0.87)
- HERA: Hybrid Explicit Representation for Ultra-Realistic Head Avatars (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.