Research map / Video / Video generation
BEAT: A Large-Scale Semantic and Emotional Multi-modal Dataset for Conversational Gestures Synthesis
Haiyang Liu, Zihao Zhu, Naoya Iwamoto, Yichen Peng, Zhengqing Li, You Zhou, Elif Bozkurt, Bo Zheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add cascaded connection (body to hands) (ablation) | CaMN · BEAT | FGD: 137.2 -> 123.7 | Improved |
| add text modality (ablation) | CaMN · BEAT | FGD: 149.4 -> 123.7 | Improved |
| add text modality (ablation) | CaMN · BEAT | BGSR: 0.171 -> 0.239 | Improved |
| add audio modality (ablation) | CaMN · BEAT | FGD: 155.7 -> 123.7 | Improved |
| add audio modality (ablation) | CaMN · BEAT | BeatAlign: 0.733 -> 0.783 | Improved |
| add speaker ID (ablation) | CaMN · BEAT | FGD: 159.1 -> 123.7 | Improved |
| add speaker ID (ablation) | CaMN · BEAT | BGSR: 0.207 -> 0.239 | Improved |
| add facial expression modality (ablation) | CaMN · BEAT | FGD: 163.3 -> 123.7 | Improved |
| add emotion modality (ablation) | CaMN · BEAT | FGD: 151.7 -> 123.7 | Improved |
| add emotion modality (ablation) | CaMN · BEAT | BGSR: 0.231 -> 0.239 | No clear effect |
| add weighted semantic loss (ablation) | CaMN · BEAT | FGD: 151.8 -> 123.7 | Improved |
| add weighted semantic loss (ablation) | CaMN · BEAT | BeatAlign: 0.786 -> 0.783 | Hurt |
Closest papers by meaning
- ConvoFusion: Multi-Modal Conversational Diffusion for Co-Speech Gesture Synthesis (CVPR 2024 · similarity 0.90)
- Retrieving Semantics from the Deep: an RAG Solution for Gesture Synthesis (CVPR 2025 · similarity 0.88)
- Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-speech Gesture Generation (CVPR 2024 · similarity 0.88)
- Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion (CVPR 2024 · similarity 0.87)
- Chain of Generation: Multi-Modal Gesture Synthesis via Cascaded Conditional Control (AAAI 2024 · similarity 0.87)
- The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
