Research map / Label-efficient and robust learning / Recognition and long-tail
Speech2Face: Learning the Face Behind a Voice
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 6-second rather than 3-second audio at test time (ablation) | Speech2Face · AVSpeech test, 5,000-image retrieval | R@1 (%): 5.86 -> 8.28 (L2); 6.22 -> 8.34 (L1); 8.54 -> 10.92 (cos) | Helped |
| Use 6-second rather than 3-second audio at test time (ablation) | Speech2Face · AVSpeech test, predicted versus true face features | L1 distance: $9.81 \pm 1.74$ -> $9.42 \pm 1.54$ | Helped |
| Train with 6-second rather than 3-second audio (ablation) | Speech2Face · AVSpeech training and reconstruction | training convergence and reconstruction quality | No clear effect |
| Add batch normalization (ablation) | Speech2Face voice encoder · AVSpeech reconstruction | facial information in reconstructions | Helped |
| Use feature-space losses instead of only pixel-level image L1 loss (ablation) | Speech2Face voice encoder · AVSpeech reconstruction | reconstruction quality | Helped |
| Add losses on the face decoder's first layer and VGG-Face's final layer (ablation) | Speech2Face voice encoder · AVSpeech training | training stability and reconstruction quality | Helped |
| Use Speech2Face audio-to-face retrieval rather than random retrieval | Speech2Face · AVSpeech test, 5,000-image retrieval | R@1 (%): 1.00 -> 5.86 (3 sec, L2); 1.00 -> 8.28 (6 sec, L2) | Helped |
Closest papers by meaning
- From Inference to Generation: End-to-end Fully Self-supervised Generation of Human Face from Speech (ICLR 2020 · similarity 0.90)
- Face Reconstruction from Voice using Generative Adversarial Networks (NeurIPS 2019 · similarity 0.90)
- From Faces to Voices: Learning Hierarchical Representations for High-quality Video-to-Speech (CVPR 2025 · similarity 0.87)
- Faces that Speak: Jointly Synthesising Talking Face and Speech from Text (CVPR 2024 · similarity 0.86)
- FaceSpeak: Expressive and High-Quality Speech Synthesis from Human Portraits of Different Styles (AAAI 2025 · similarity 0.86)
- ProsodyTalker: 3D Visual Speech Animation via Prosody Decomposition (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.