Research map / Label-efficient and robust learning / Recognition and long-tail
Low-Rank Head Avatar Personalization with Registers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Register Module that learns a 3D feature space on a 3DMM mesh to augment LoRA personalization of the GAGAvatar head avatar model, capturing identity-specific high-frequency details such as wrinkles and tattoos.
Problem
Generic head avatar models trained on multi-identity data miss identity-specific high-frequency facial details, and vanilla LoRA personalization does not sufficiently recover them.
Approach
A learnable 3D register embedding rigged to 3DMM mesh vertices is projected to the image plane for visible vertices, interpolated over the face, and added to DINOv2 features during LoRA adaptation only; it is discarded at inference, extending ViT registers to 3D faces.
Key results
- On RareFace-50, the method reports LPIPS 0.2744 and ACD 0.3792, versus 0.2855 / 0.3872 for LoRA and 0.2901 / 0.4504 for MetaPortrait-style meta-learning on LoRA.
- On VFHQ Test, the method reports LPIPS 0.2470 and ACD 0.3559, the best of the compared methods (LoRA: 0.2666 / 0.3687).
- In a user study with 18 participants, 78.2% preferred the method over LoRA for identity preservation and 89.9% for visual quality.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the Register Module with Gaussian noise added to DINOv2 features during adaptation (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | LPIPS: 0.2470 -> 0.2699 | Hurt |
| replace the Register Module with Gaussian noise added to DINOv2 features during adaptation (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | ACD: 0.3559 -> 0.3795 | Hurt |
| replace the Register Module with learnable embeddings added to DINOv2 features during adaptation (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | LPIPS: 0.2470 -> 0.2622 | Hurt |
| replace the Register Module with learnable embeddings added to DINOv2 features during adaptation (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | ACD: 0.3559 -> 0.3604 | Hurt |
| add L_feat supervision during adaptation (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | LPIPS: 0.2574 -> 0.2470 | Helped |
| add L_feat supervision during adaptation (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | ACD: 0.3604 -> 0.3559 | Helped |
| add L_reg regularization during adaptation (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | LPIPS: 0.2508 -> 0.2470 | Helped |
| add L_reg regularization during adaptation (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | ACD: 0.3574 -> 0.3559 | Helped |
| reduce adaptation video length from 4 seconds to 2 seconds (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | LPIPS: 0.2471 -> 0.2471 | No clear effect |
| reduce adaptation video length from 4 seconds to 2 seconds (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | ACD: 0.3639 -> 0.3656 | Hurt |
| reduce adaptation video length from 4 seconds to 1 second (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | LPIPS: 0.2471 -> 0.2477 | Hurt |
| reduce adaptation video length from 4 seconds to 1 second (ablation) | LoRA personalization with GAGAvatar · VFHQ Test | ACD: 0.3639 -> 0.3687 | Hurt |
Surprising findings
- Meta-learning on LoRA (MetaPortrait-style) performs worse than plain LoRA on both benchmarks, with higher ACD.
Limitations
- Performance may degrade for extreme side or back views rarely seen in the source video.
- Results depend on noisy 3DMM fitting, which does not model asymmetric expressions or tongue movement.
- Evaluated on a small set of 50 identities in the new dataset and 50 VFHQ test videos.
Experimental setup
Datasets: RareFace-50 VFHQ
Benchmarks: VFHQ Test RareFace-50
Metrics: LPIPS ACD user preference
Baselines: GAGAvatar LoRA MetaPortrait (meta-learning on LoRA) Gaussian noise register variant learnable embeddings variant
Builds on: LoRA GAGAvatar DINOv2 MetaPortrait minLoRA
Closest papers by meaning
- Gaussian Eigen Models for Human Heads (CVPR 2025 · similarity 0.88)
- PERSE: Personalized 3D Generative Avatars from A Single Portrait (CVPR 2025 · similarity 0.87)
- Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance (CVPR 2025 · similarity 0.87)
- Expressive Gaussian Human Avatars from Monocular RGB Video (NeurIPS 2024 · similarity 0.87)
- Synthetic Prior for Few-Shot Drivable Head Avatar Inversion (CVPR 2025 · similarity 0.87)
- PEGASUS: Personalized Generative 3D Avatars with Composable Attributes (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.