Research map / 3D vision / Face and shape
ScanTalk: 3D Talking Heads from Unregistered Scans
Federico Nocentini, Thomas Besnier, Claudio Ferrari, Sylvain Arguillere, Stefano Berretti, Mohamed Daoudi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HuBERT audio encoder instead of WavLM (ablation) | ScanTalk · VOCAset (single-dataset) | LVE: 3.674 -> 3.012 | Improved |
| use HuBERT audio encoder instead of Wav2Vec2 (ablation) | ScanTalk · VOCAset (multi-dataset) | LVE: 7.127 -> 6.375 | Improved |
| add BiLSTM temporal consistency module (ablation) | ScanTalk · VOCAset (single-dataset) | LVE: 3.361 -> 3.012 | Improved |
| use BiLSTM instead of BiGRU (ablation) | ScanTalk · VOCAset (single-dataset) | LVE: 3.036 -> 3.012 | No clear effect |
| use BiLSTM instead of Transformer Decoder (ablation) | ScanTalk · VOCAset (multi-dataset) | LVE: 6.512 -> 6.375 | Improved |
| add masked and velocity losses to plain L2 loss (ablation) | ScanTalk (multi-dataset) · VOCAset (multi-dataset) | LVE: 6.375 -> 7.031 | Hurt |
| add masked loss to plain L2 loss (ablation) | ScanTalk (multi-dataset) · VOCAset (multi-dataset) | LVE: 6.375 -> 7.451 | Hurt |
| train on multiple datasets instead of VOCAset only | ScanTalk · VOCAset | LVE: 3.012 -> 6.375 | Hurt |
| train on multiple datasets instead of BIWI_6 or Multiface only | ScanTalk · BIWI_6 and Multiface | error metrics | Improved |
| add surface normals to input positions (ablation) | ScanTalk mesh encoding · supplementary | quality | No clear effect |
| add Heat Kernel Signature features (ablation) | ScanTalk mesh encoding · supplementary | quality | No clear effect |
Closest papers by meaning
- FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models (CVPR 2024 · similarity 0.89)
- Perceptually Accurate 3D Talking Head Generation: New Definitions, Speech-Mesh Representation, and Evaluation Metrics (CVPR 2025 · similarity 0.88)
- Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animation (AAAI 2024 · similarity 0.88)
- AnyTalk: Multi-modal Driven Multi-domain Talking Head Generation (AAAI 2025 · similarity 0.88)
- Probabilistic Speech-Driven 3D Facial Motion Synthesis: New Benchmarks Methods and Applications (CVPR 2024 · similarity 0.87)
- Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
