Research map / 3D vision / Face and shape
AVFace: Towards Detailed Audio-Visual 4D Face Reconstruction
Aggelina Chatziagapi, Dimitris Samaras
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add temporal modeling to video-only expression and jaw prediction (ablation) | AVFace expression-jaw branch · FaceSet | NME (%): 2.60 -> 1.98 | Improved |
| Add temporal modeling to audio-only expression and jaw prediction (ablation) | AVFace expression-jaw branch · FaceSet | NME (%): 2.80 -> 2.10 | Improved |
| Add modality dropout to the AV expression-jaw branch (ablation) | AVFace · FaceSet | NME (%): 2.91 -> 2.14 | Improved |
| Add temporal modeling to AV expression-jaw prediction with modality dropout (ablation) | AVFace · FaceSet | NME (%): 2.14 -> 1.73 | Improved |
| Add lip refinement to the AV expression-jaw model with dropout and temporal modeling (ablation) | AVFace · FaceSet | NME (lips) (%): 0.99 -> 0.91 | Improved |
| Use DeepSpeech instead of wav2vec 2.0 audio features (ablation) | AVFace · FaceSet | NME (lips) (%): 1.11 -> 0.91 | Improved |
| Use DeepSpeech instead of mel-spectrogram audio features (ablation) | AVFace · FaceSet | NME (%): 2.20 -> 1.63 | Improved |
| Fine-tune the coarse stage on synthetic face occlusions (ablation) | AVFace · Synthetic test set with occlusions | NME (occ) (%): 2.62 -> 2.39 | Improved |
| Use AVFace instead of 3DDFA-V2 | Face reconstruction methods · FaceSet | NME (lips) (%): 1.15 -> 0.91 | Improved |
| Use AVFace instead of DECA | Face reconstruction methods · Synthetic test set with occlusions | NME (occ) (%): 3.89 -> 2.39 | Improved |
| Use AVFace instead of DECA | 3D face reconstruction methods · Multiface | Mean 3D reconstruction error (mm): 2.01 -> 1.98 | Improved |
Closest papers by meaning
- Speech4Mesh: Speech-Assisted Monocular 3D Facial Reconstruction for Speech-Driven 3D Facial Animation (ICCV 2023 · similarity 0.90)
- MeshTalk: 3D Face Animation From Speech Using Cross-Modality Disentanglement (ICCV 2021 · similarity 0.88)
- Face Anything: 4D Face Reconstruction from Any Image Sequence (ECCV 2026 · similarity 0.88)
- Semi-supervised Speech-driven 3D Facial Animation via Cross-modal Encoding (ICCV 2023 · similarity 0.88)
- HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details (ICCV 2023 · similarity 0.87)
- Probabilistic Speech-Driven 3D Facial Motion Synthesis: New Benchmarks Methods and Applications (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
