Research map / 3D vision / Face and shape
MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MeGA, a hybrid head avatar that models the face with an enhanced FLAME mesh plus deferred neural rendering and the hair with rigged 3D Gaussians, fused by occlusion-aware blending.
Problem
High-fidelity animatable head avatars from multi-view video struggle to render all head components well with one representation, since hair (volumetric thin structures) and skin (surface-like, low-dimensional) have different characteristics.
Approach
Decompose the head into a neural-mesh face (FLAME subdivided, UV displacement map, disentangled diffuse/view/dynamic neural textures decoded by a tiny per-pixel MLP) and a canonical 3DGS hair animated by ICP rigid transform plus an MLP deformation field, blended with a near-z occlusion test, early-stopped Gaussian rendering, and soft-edge masks.
Key results
- On NeRSemble novel-view synthesis, MeGA reaches 34.11 PSNR, 0.954 SSIM, 0.052 LPIPS, versus 33.54 PSNR for GaussianAvatars.
- On novel-expression synthesis, MeGA reaches 32.59 PSNR and 0.057 LPIPS, versus 31.45 PSNR and 0.060 LPIPS for GaussianAvatars.
- Ablation on subject 306: replacing near-z depth with alpha-accumulated GS depth drops PSNR from 33.57 to 27.94.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MeGA's hybrid mesh-Gaussian representation instead of GaussianAvatars | NeRSemble head avatars · NeRSemble novel-view synthesis | PSNR: 33.54 -> 34.11 | Helped |
| Use MeGA's hybrid mesh-Gaussian representation instead of GaussianAvatars | NeRSemble head avatars · NeRSemble novel-expression synthesis | PSNR: 31.45 -> 32.59 | Helped |
| Add view texture (ablation) | MeGA · Subject 306 | PSNR: 31.68 -> 33.57 | Helped |
| Add expression-dependent dynamic texture (ablation) | MeGA · Subject 306 | PSNR: 32.81 -> 33.57 | Helped |
| Add mesh subdivision (ablation) | MeGA · Subject 306 | PSNR: 32.81 -> 33.57 | Helped |
| Add UV displacement map (ablation) | MeGA · Subject 306 | PSNR: 32.99 -> 33.57 | Helped |
| Use near-z depths rather than alpha-accumulated depths for occlusion testing (ablation) | MeGA · Subject 306 | PSNR: 27.94 -> 33.57 | Helped |
| Use early-stopping during Gaussian hair rendering (ablation) | MeGA · Subject 306 | PSNR: 31.42 -> 33.57 | Helped |
Surprising findings
- Using 3DGS-rendered depth for the occlusion test makes training unstable and causes a large PSNR drop (33.57 to 27.94).
- Gaussian Head Avatar, strong on novel-view synthesis (29.48 PSNR), performs poorly on novel expressions (22.02 PSNR).
Limitations
- Evaluation limited to the NeRSemble dataset with 9 subjects.
- Head editing is shown only qualitatively, since no prior methods are directly comparable.
Experimental setup
Datasets: NeRSemble
Benchmarks: NeRSemble
Metrics: PSNR SSIM LPIPS geometry MAE
Baselines: GaussianAvatars Gaussian Head Avatar PointAvatar DELTA Pixel Codec Avatars
Builds on: 3D Gaussian Splatting Pixel Codec Avatars GaussianAvatars deferred neural rendering FLAME
Closest papers by meaning
- HERA: Hybrid Explicit Representation for Ultra-Realistic Head Avatars (CVPR 2025 · similarity 0.91)
- Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians (CVPR 2024 · similarity 0.91)
- LUCAS: Layered Universal Codec Avatars (CVPR 2025 · similarity 0.91)
- GPAvatar: High-fidelity Head Avatars by Learning Efficient Gaussian Projections (CVPR 2025 · similarity 0.90)
- 3D Gaussian Head Avatars with Expressive Dynamic Appearances by Compact Tensorial Representations (CVPR 2025 · similarity 0.90)
- GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.