Research map / 3D vision / Face and shape
GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Edits a single rendered image of a volumetric 3DMM-driven head avatar by lifting 2D edits into an expression-aware 3D modification field, trained via distillation from a 3DMM-based GAN (Next3D) and 2D makeup/texture tools.
Problem
Editing animatable volumetric head avatars (NeRF-based, various representations) consistently across expressions and viewpoints from a single edited image, which existing methods do not support in a representation-agnostic way.
Approach
An expression-aware modification generator (tri-plane modification fields conditioned on 3DMM expression and latent codes) that is representation-agnostic, trained by distilling geometry and texture edits from a teacher 3DMM GAN and 2D editing tools, then fitted to a single user-edited image by auto-decoding latent optimization.
Key results
- User study: ours preferred in 79.44% overall for texture editing and 43.33% overall (vs. PVP 23.33%, Next3D 29.44%-ish range) for geometry editing per Table 1.
- Image identity similarity of 0.8845 for geometry editing (ours) versus 0.8373-0.8704 for baselines.
- Evaluated on 19 neural head avatars from INSTA (7), NeRFBlendShape (8), and Next3D (4).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GeneAvatar instead of Roop | Single-view volumetric head avatar geometry editing · Geometry editing user study | Editing preservation: 29.44% -> 41.67% | Helped |
| Use GeneAvatar instead of Roop | Single-view volumetric head avatar geometry editing · Geometry editing user study | Identity preservation: 31.11% -> 42.22% | Helped |
| Use GeneAvatar instead of Roop | Single-view volumetric head avatar geometry editing · Geometry editing user study | Temporal consistency: 30.00% -> 41.67% | Helped |
| Use GeneAvatar instead of Roop | Single-view volumetric head avatar geometry editing · Geometry editing user study | Overall: 29.44% -> 43.33% | Helped |
| Use GeneAvatar instead of PVP | Single-view volumetric head avatar geometry editing · Geometry editing | Image identity similarity: 0.8704 -> 0.8845 | Helped |
| Use GeneAvatar instead of PVP | Single-view volumetric head avatar texture editing · Texture editing user study | Editing preservation: 10.56% -> 76.11% | Helped |
| Use GeneAvatar instead of PVP | Single-view volumetric head avatar texture editing · Texture editing user study | Identity preservation: 12.78% -> 76.11% | Helped |
| Use GeneAvatar instead of PVP | Single-view volumetric head avatar texture editing · Texture editing user study | Temporal consistency: 12.78% -> 80.00% | Helped |
| Use GeneAvatar instead of PVP | Single-view volumetric head avatar texture editing · Texture editing user study | Overall: 12.78% -> 79.44% | Helped |
| Use GeneAvatar instead of Next3D | Single-view volumetric head avatar texture editing · Texture editing | Image identity similarity: 0.8500 -> 0.9147 | Helped |
Surprising findings
- Direct 3DMM shape-parameter updates from single-image reconstruction fail for out-of-distribution edits and produce blurry renderings even after decoder fine-tuning.
Limitations
- Cannot add new objects (e.g., hats) or modify hairstyle.
- Relies on teacher Next3D GAN and 2D editing tools for distillation data.
- 3DMM fitting fails for out-of-distribution edits in the baseline comparison.
Experimental setup
Datasets: Next3D-generated synthetic avatars makeup transfer dataset INSTA monocular head videos NeRFBlendShape head data
Benchmarks: user study on 12 groups of edits (30 participants) image identity similarity on 350 rendered images
Metrics: user preference percentage (editing preservation, identity p cosine image identity similarity
Baselines: PVP Roop Next3D 3DMM fitting (Deep3DFace-style single-image reconstruction) fine-tuning NeRF
Builds on: Next3D NeRFBlendShape INSTA PTI StyleGAN
Closest papers by meaning
- Synthetic Prior for Few-Shot Drivable Head Avatar Inversion (CVPR 2025 · similarity 0.88)
- GPAvatar: Generalizable and Precise Head Avatar from Image(s) (ICLR 2024 · similarity 0.88)
- HRAvatar: High-Quality and Relightable Gaussian Head Avatar (CVPR 2025 · similarity 0.88)
- MeGA: Hybrid Mesh-Gaussian Head Avatar for High-Fidelity Rendering and Head Editing (CVPR 2025 · similarity 0.88)
- High-Fidelity 3D Head Avatars Reconstruction through Spatially-Varying Expression Conditioned Neural Radiance Field (AAAI 2024 · similarity 0.88)
- RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatars (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.