Research map / Video / Video generation
InterActHuman: Multi-Concept Human Animation with Layout-Aligned Audio Conditions
Zhenzhi Wang, Jiaqi Yang, Jianwen Jiang, Chao Liang, Gaojie Lin, Zerong Zheng, Ceyuan Yang, Yuan Zhang, Mingyuan Gao, Dahua Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add predicted-mask local audio injection instead of global audio (ablation) | InterActHuman · Multi-Person Test Set (full-body) | Sync-D: 9.482 -> 6.670 | Improved |
| add predicted-mask injection instead of global audio (ablation) | InterActHuman · Multi-Person Test Set (full-body) | FVD: 33.895 -> 22.881 | Improved |
| add predicted dynamic mask instead of fixed mask (ablation) | InterActHuman · Multi-Person Test Set (full-body) | FVD: 40.239 -> 22.881 | Improved |
| add predicted dynamic mask instead of fixed mask (ablation) | InterActHuman · Multi-Person Test Set (full-body) | Sync-D: 7.068 -> 6.670 | Improved |
| add explicit predicted mask instead of learnable ID embedding (ablation) | InterActHuman · Multi-Person Test Set (full-body) | Sync-D: 8.627 -> 6.670 | Improved |
| add explicit predicted mask instead of learnable ID embedding (ablation) | InterActHuman · Multi-Person Test Set (full-body) | FVD: 35.665 -> 22.881 | Improved |
| add predicted mask instead of global audio (ablation) | InterActHuman · Multi-Person Test Set (full-body) | IQA: 4.768 -> 4.757 | No clear effect |
| add mask cache from previous denoising step (ablation) | InterActHuman · Multi-person lip sync (high motion) | Sync-D: 11.046 -> 6.921 | Improved |
| combine masks across layers instead of layer 4 only (step 50) (ablation) | InterActHuman · Low motion mask IoU | mask IoU: 0.858 -> 0.956 | Improved |
| more denoising steps for mask refinement (step 1 to step 50) | InterActHuman combined mask · Low motion mask IoU | mask IoU: 0.376 -> 0.956 | Improved |
| more denoising steps for mask refinement (step 1 to step 50) | InterActHuman combined mask · High motion mask IoU | mask IoU: 0.741 -> 0.937 | Improved |
| scale from <=3 to 4-5 people | InterActHuman · Scaling beyond three people | Sync-D: 6.670 -> 6.608 | No clear effect |
Closest papers by meaning
- Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation (NeurIPS 2025 · similarity 0.89)
- FaceChain-ImagineID: Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled Audio (CVPR 2024 · similarity 0.86)
- AnyTalk: Multi-modal Driven Multi-domain Talking Head Generation (AAAI 2025 · similarity 0.86)
- EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation (CVPR 2025 · similarity 0.86)
- VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time (NeurIPS 2024 · similarity 0.85)
- Synergizing Motion and Appearance: Multi-Scale Compensatory Codebooks for Talking Head Video Generation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
