Research map / 3D vision / 3D generation
Bridging Facial Understanding and Animation via Language Models
Luchuan Song, Pinxin Liu, Haiyang Liu, Zhenchao Jin, Yolo Yunlong Tang, Zichong Xu, Susan Liang, Jing Bi, Jason J Corso, Chenliang Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on Open3DFaceVid instead of MEAD (ablation) | Motion2Language · Motion2Language | Cor_E (GPT-4): 3.03 -> 3.97 | Improved |
| train on Open3DFaceVid instead of YouTube (ablation) | Motion2Language · Motion2Language | Cor_M (GPT-4): 3.92 -> 3.44 | Hurt |
| train on Open3DFaceVid instead of YouTube (ablation) | Motion2Language · Motion2Language | Cor_E (GPT-4): 2.74 -> 3.97 | Improved |
| train on Open3DFaceVid instead of MEAD (ablation) | Language2Motion · Language2Motion | USER: 2.88 -> 3.92 | Improved |
| train on Open3DFaceVid instead of MEAD (ablation) | Language2Motion · Language2Motion | FD: 32.14 -> 32.48 | Mixed |
| train on Open3DFaceVid instead of MEAD (ablation) | Language2Motion · Language2Motion | Cor_E: 3.56 -> 3.90 | Improved |
| use geometry tokens in an LLM instead of image-token VLMs | Motion2Language · Motion2Language (geometry input) | Cor_E (GPT-4): 2.45 -> 4.02 vs Gemini2.5 | Improved |
| use geometry tokens vs Gemini on natural images | Motion2Language · natural image input | Cor_E (GPT-4): 4.21 (Gemini) vs 4.02 (Ours) | Hurt |
| condition on LLM word-level prefix instead of GPT-style text conditioning | Language2Motion · Language2Motion | Cor_E: 3.57 -> 4.13 | Improved |
| scale Motion2Language backbone beyond 4B-8B (ablation) | Qwen3 backbones · Motion2Language | token accuracy | No clear effect |
| scale Language2Motion backbone from 0.7B to 3B (ablation) | LLaMA backbones · Language2Motion | token accuracy | Improved |
Closest papers by meaning
- DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation (AAAI 2025 · similarity 0.89)
- The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion (CVPR 2025 · similarity 0.89)
- ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignment (AAAI 2024 · similarity 0.88)
- Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animation (AAAI 2024 · similarity 0.88)
- KeyFace: Expressive Audio-Driven Facial Animation for Long Sequences via KeyFrame Interpolation (CVPR 2025 · similarity 0.87)
- Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
