Research map / 3D vision / Face and shape
HRAvatar: High-Quality and Relightable Gaussian Head Avatar
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HRAvatar, a 3DGS-based monocular head avatar method with end-to-end expression encoder, learnable blendshapes and skinning, and physically-based shading for real-time relightable avatars.
Problem
Reconstructing high-quality, animatable 3D head avatars with realistic relighting from monocular video under unknown lighting, where prior 3DGS avatar methods suffer from inaccurate expression tracking, limited deformation flexibility, and cannot relight.
Approach
Trains an expression encoder end-to-end with photometric loss instead of fixed pre-tracked expressions, learns per-Gaussian blendshape bases and skinning weights initialized from FLAME, and decomposes appearance into albedo, roughness, and Fresnel reflectance with split-sum physically-based shading under optimizable environment maps.
Key results
- On the INSTA dataset self-reenactment, HRAvatar reaches 30.36 PSNR, 0.9482 SSIM and 0.0569 LPIPS, beating GBS (29.64 PSNR, 0.0823 LPIPS).
- Renders and relights at about 155 FPS.
- Ablation: removing the albedo prior or normal loss yields unrealistic relighting; replacing learnable deformation with FLAME rigging drops PSNR from 30.36 to 29.79.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HRAvatar instead of GBS | Head avatar reconstruction · INSTA dataset | PSNR: 29.64 -> 30.36 | Helped |
| Use HRAvatar instead of GBS | Head avatar reconstruction · HDTF dataset | PSNR: 27.81 -> 28.55 | Helped |
| Use HRAvatar instead of GBS | Head avatar reconstruction · self-captured dataset | PSNR: 28.59 -> 28.97 | Helped |
| Add learnable per-point blendshapes and blend weights (ablation) | HRAvatar · INSTA dataset | PSNR: 29.83 -> 30.36 | Helped |
| Use learnable blendshapes and LBS instead of FLAME-rigging (ablation) | HRAvatar · INSTA dataset | PSNR: 29.79 -> 30.36 | Helped |
| Use per-point learnable deformation instead of a shared MLP (ablation) | HRAvatar · INSTA dataset | PSNR: 29.67 -> 30.36 | Helped |
| Add the end-to-end expression encoder instead of using pre-tracked parameters (ablation) | HRAvatar · INSTA dataset | PSNR: 29.70 -> 30.36 | Helped |
| Use physically-based shading instead of standard 3DGS appearance (ablation) | HRAvatar · INSTA dataset | PSNR: 30.34 -> 30.36 | No clear effect |
| Add normal consistency loss (ablation) | HRAvatar · Relighting | visual relighting quality | Helped |
| Add the albedo prior loss (ablation) | HRAvatar · Relighting | visual relighting quality | Helped |
Limitations
- Constrained by FLAME priors when training data is insufficient, affecting hair or accessories.
- Shadows or wrinkles may be mis-coupled into albedo or reflectance, hurting relighting of specular reflections and shadows.
- Full-head reconstruction from monocular video is infeasible with unknown camera poses when yaw approaches 90 degrees.
Experimental setup
Datasets: INSTA HDTF self-captured mobile phone videos
Benchmarks: INSTA HDTF self-captured dataset
Metrics: PSNR MAE SSIM LPIPS
Baselines: Point-avatar INSTA Splatting-avatar Flash-avatar 3D Gaussian Blendshapes (GBS) FLARE
Builds on: 3D Gaussian Splatting FLAME SMIRK Point-avatar SplitSum
Closest papers by meaning
- RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatars (CVPR 2025 · similarity 0.92)
- GPAvatar: High-fidelity Head Avatars by Learning Efficient Gaussian Projections (CVPR 2025 · similarity 0.91)
- Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians (CVPR 2024 · similarity 0.90)
- GraphAvatar: Compact Head Avatars with GNN-Generated 3D Gaussians (AAAI 2025 · similarity 0.89)
- GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians (CVPR 2024 · similarity 0.89)
- Generalizable and Animatable Gaussian Head Avatar (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.