Research map / Video / Video generation
She Performs Your Voice: A Unified Speech and Dance Motion Model
Lurui Wang, yongkang cheng, Can Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a unified audio-driven model that generates both speech-synchronized gestures and music-driven dance motion from a shared audio representation.
Problem
Existing audio-driven character systems treat audio as a task-specific condition, optimizing separately for co-speech gestures and music-driven dance, which fragments the shared perceptual link between sound and motion.
Approach
Treats audio as the shared perceptual driver of performance motion, unifying speech-driven gesture and music-driven dance generation in one model rather than separate task-specific systems.
Closest papers by meaning
- 🎧MOSPA: Human Motion Generation Driven by Spatial Audio (NeurIPS 2025 · similarity 0.89)
- Text Dictates, Music Decorates: Energy-based Attention for Editable Dance Generation (ECCV 2026 · similarity 0.89)
- TMD-Bench: A Multi-Level Evaluation Paradigm for Music–Dance Co-Generation (ICML 2026 · similarity 0.88)
- Do You Have Freestyle? Expressive Humanoid Locomotion via Audio Control (CVPR 2026 · similarity 0.88)
- Odoriko: A Shape-Aware Multimodal Diffusion Framework for Human Motion (ECCV 2026 · similarity 0.88)
- Sonic: Shifting Focus to Global Audio Perception in Portrait Animation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
