Research map / 3D vision / Face and shape
GLVD: Guided Learned Vertex Descent
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GLVD, a hybrid few-shot 3D face reconstruction method that extends Learned Vertex Descent with per-vertex neural fields guided by dynamically predicted 3D keypoints and relative keypoint encodings.
Problem
Few-shot 3D face reconstruction from one or more images: 3DMM-based methods are limited to fixed shape priors, optimization methods are expensive, and feed-forward pixel-aligned methods lag in accuracy.
Approach
Iteratively refines mesh vertices with per-vertex neural field displacements conditioned on pixel-aligned features and on vertex positions encoded relative to dynamically predicted 3D keypoints, which provide global structural guidance without dense 3D supervision or parametric priors.
Key results
- Single-view 3DFAW Chamfer error of 1.25 mm, versus 1.58 for LVD and 1.45 for 3DDFA-V3 (best among compared methods).
- Three-view 3DFAW error of 1.22 mm, and H3DS 2.0 single-view error of 1.36 mm, compared with 1.39 for LVD.
- Reported to achieve comparable accuracy to optimization-based methods with over two orders of magnitude faster inference, with 10 update steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add dynamically predicted keypoint guidance to LVD | LVD · 3DFAW, 1 view | surface error (mm): 1.58 -> 1.25 | Helped |
| Add dynamically predicted keypoint guidance to LVD | LVD · H3DS 2.0, 1 view | surface error (mm): 1.39 -> 1.36 | Helped |
| Increase the number of keypoints from 4 to 18 (ablation) | GLVD · 3DFAW, 1 view | Chamfer distance (mm): 1.92 -> 1.25 | Helped |
| Increase the number of keypoints from 18 to 24 (ablation) | GLVD · 3DFAW, 1 view | Chamfer distance (mm): 1.25 -> 1.58 | Hurt |
| Use 18 rather than 4 keypoints (ablation) | GLVD · H3DS 2.0, 1 view | Chamfer distance (mm): 1.51 -> 1.36 | Helped |
| Remove HRNet (ablation) | GLVD · 3DFAW, 1 view | Chamfer distance (mm): 1.25 -> 1.52 | Hurt |
| Add training noise to keypoints (ablation) | GLVD · 3DFAW, 1 view | Chamfer distance (mm): 1.31 -> 1.25 | Helped |
| Add binary dropout to image features (ablation) | GLVD · 3DFAW, 1 view | Chamfer distance (mm): 1.27 -> 1.25 | Helped |
| Optimize in canonical space (ablation) | GLVD · 3DFAW, 1 view | Chamfer distance (mm): 1.25 -> 1.51 | Hurt |
| Remove absolute vertex positions and use only relative encoding (ablation) | GLVD · H3DS 2.0, 1 view | Chamfer distance (mm): 1.36 -> 1.67 | Hurt |
| Replace relative encoding with concatenation encoding (ablation) | GLVD · H3DS 2.0, 1 view | Chamfer distance (mm): 1.36 -> 1.36 | No clear effect |
| Replace relative encoding with normalized-distance encoding (ablation) | GLVD · 3DFAW, 1 view | Chamfer distance (mm): 1.26 -> 1.25 | No clear effect |
Surprising findings
- Optimizing in canonical space, as in prior work, introduces a bias toward the mean shape and hurts accuracy compared with camera coordinates in the single-view setting.
- Predicting a specific vertex directly via identity features was found less stable than implicit trajectory encoding.
Limitations
- Sensitive to occlusions and relies on keypoint prediction accuracy; performance degrades at 45 and 90 degree viewing angles (2.07 and 2.11 mm).
- Focuses on face region only and does not model facial expressions.
- Multi-view results are only competitive, not state-of-the-art, relative to optimization methods.
Experimental setup
Datasets: proprietary 3D head scan dataset (10,000 individuals) H3DS 2.0 3DFAW CelebA-HQ
Benchmarks: 3DFAW H3DS 2.0 CelebA-HQ
Metrics: unidirectional Chamfer distance (mm) inference time
Baselines: MVFNet DFNRMVS DECA MICA FaceVerse FaceScape HRN VHAP 3DDFA-V3 RAFaRe PIFu JIFF
Builds on: Learned Vertex Descent HRNet stacked hourglass network KeypointNeRF
Closest papers by meaning
- Uncertainty-Aware Mesh Decoder for High Fidelity 3D Face Reconstruction (CVPR 2020 · similarity 0.86)
- Regressing Robust and Discriminative 3D Morphable Models With a Very Deep Neural Network (CVPR 2017 · similarity 0.86)
- MVF-Net: Multi-View 3D Face Morphable Model Regression (CVPR 2019 · similarity 0.86)
- Learning To Restore 3D Face From In-the-Wild Degraded Images (CVPR 2022 · similarity 0.86)
- G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Images (CVPR 2024 · similarity 0.84)
- 3D Facial Expressions through Analysis-by-Neural-Synthesis (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.