Research map / 3D vision / Face and shape
Efficient 3D Implicit Head Avatar with Mesh-anchored Hash Table Blendshapes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes mesh-anchored hash table blendshapes, attaching multiple per-vertex hash tables to a FLAME mesh, linearly merged by a UV-space CNN, plus hierarchical k-NN search, for real-time monocular 3D neural head avatars.
Problem
Neural implicit head avatars achieve high photorealism but are too slow for real-time rendering, and fast hash-encoding NeRFs designed for static scenes do not easily support controllable facial expressions.
Approach
Attach multiple small hash tables to each 3DMM (FLAME) vertex as local blendshapes, linearly merge them with expression-dependent weights predicted by a UV-space U-Net, and decode k-NN-interpolated hash embeddings with a tiny two-hidden-layer MLP; a voxel-grouped hierarchical k-NN search further speeds up rendering.
Key results
- Achieves 35.9 mean FPS at 512x512 on RTX3090Ti with LPIPS 0.100, SSIM 0.795, PSNR 22.77, matching MonoAvatar quality (LPIPS 0.114, PSNR 22.74) at about 70x its speed (0.5 FPS).
- Hierarchical k-NN search improves speed from 26.4 to 35.9 FPS (~36%) without visible quality loss at 64 grid resolution.
- Using 5 hash table blendshapes per vertex improves LPIPS from 0.125 (static hash + 3DMM params) to 0.100 and saturates with more tables.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use mesh-anchored hash table blendshapes rather than a single static hash table with 3DMM codes (ablation) | 3DMM-anchored head avatar · 10-video test set | LPIPS: 0.125 -> 0.100 | Helped |
| add UV CNN-predicted expression-dependent blendshape weights to the static hash tables (ablation) | 3DMM-anchored head avatar · 10-video test set | LPIPS: 0.115 -> 0.100 | Helped |
| increase the number of hash table blendshapes per vertex from 3 to 5 (ablation) | mesh-anchored hash table avatar · 10-video test set | LPIPS: 0.104 -> 0.100 | Helped |
| use hierarchical k-NN search instead of searching all vertices for each query point (ablation) | mesh-anchored hash table avatar · 10-video test set | Rendering speed: 26.4 FPS -> 35.9 FPS | Helped |
| use 64 rather than 32 or 16 grid resolution for hierarchical k-NN search (ablation) | mesh-anchored hash table avatar · 10-video test set | Rendering quality | Helped |
| use the proposed avatar instead of INSTA | head avatar rendering · 10-video test set | LPIPS: 0.149 -> 0.100 | Helped |
| use the proposed avatar instead of MonoAvatar | head avatar rendering · 10-video test set | Mean FPS: 0.5 -> 35.9 | Helped |
| use the proposed avatar instead of NeRFBlendshape | head avatar rendering · 10-video test set | Mean FPS: 11.2 -> 35.9 | Helped |
| use the proposed avatar instead of PointAvatar | head avatar rendering · 10-video test set | LPIPS: 0.117 -> 0.100 | Helped |
Surprising findings
- Static hash with 3DMM parameter inputs without a UV CNN yields the worst quality, showing the MLP lacks capacity for expression detail from compact codes.
Limitations
- Floaters appear under camera viewpoints and expressions far from the training distribution, a common issue in Instant NGP-based methods.
- Mouth interior regions are less stable due to poor tracking in training data.
- Evaluation uses only 10 monocular videos from a limited set of subjects.
Experimental setup
Datasets: monocular RGB videos from PointAvatar, INSTA, MonoAvatar, plus self-captured videos
Benchmarks: 10 monocular head avatar videos (held-out clips)
Metrics: PSNR SSIM LPIPS mean FPS GFLOPs
Baselines: NeRFBlendshape INSTA PointAvatar MonoAvatar
Builds on: MonoAvatar Instant NGP NeRFBlendshape Nerfies FLAME
Closest papers by meaning
- HERA: Hybrid Explicit Representation for Ultra-Realistic Head Avatars (CVPR 2025 · similarity 0.90)
- DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars (CVPR 2024 · similarity 0.90)
- Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians (CVPR 2024 · similarity 0.89)
- High-Fidelity 3D Head Avatars Reconstruction through Spatially-Varying Expression Conditioned Neural Radiance Field (AAAI 2024 · similarity 0.89)
- MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing (CVPR 2025 · similarity 0.89)
- 3D Gaussian Head Avatars with Expressive Dynamic Appearances by Compact Tensorial Representations (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.