Research map / Video / Video generation
DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generation
Xu Guo, Fulong Ye, Qichao Sun, Liyang Chen, Bingchuan Li, Pengze Zhang, Jiawei Liu, Songtao Zhao, Qian HE, Xiangwang Hou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Structured Caption (ablation) | DreamID-Omni · IDBench-Omni R2AV (multi-person ablation) | Spk-Conf (lower better): 0.26 -> 0.08 | Improved |
| add Structured Caption (ablation) | DreamID-Omni · IDBench-Omni R2AV (multi-person ablation) | ViCLIP: 11.381 -> 13.613 | Improved |
| add Syn-RoPE (ablation) | DreamID-Omni · IDBench-Omni R2AV (multi-person ablation) | T-Sim: 0.211 -> 0.402 | Improved |
| add Syn-RoPE (ablation) | DreamID-Omni · IDBench-Omni R2AV (multi-person ablation) | Sync-C: 4.192 -> 6.074 | Improved |
| add Syn-RoPE (ablation) | DreamID-Omni · IDBench-Omni R2AV (multi-person ablation) | Spk-Conf (lower better): 0.12 -> 0.08 | Improved |
| add cross-pair disentanglement after in-pair reconstruction (ablation) | DreamID-Omni · IDBench-Omni R2AV (single-person ablation) | ViCLIP: 11.931 -> 14.573 | Improved |
| add cross-pair disentanglement after in-pair reconstruction (ablation) | DreamID-Omni · IDBench-Omni R2AV (single-person ablation) | ID-Sim: 0.692 -> 0.674 | Hurt |
| add in-pair reconstruction stage before cross-pair (ablation) | DreamID-Omni · IDBench-Omni R2AV (single-person ablation) | ID-Sim: 0.543 -> 0.674 | Improved |
| add in-pair reconstruction stage before cross-pair (ablation) | DreamID-Omni · IDBench-Omni R2AV (single-person ablation) | T-Sim: 0.471 -> 0.493 | Improved |
| add progressive omni-task fine-tuning stage instead of joint multi-task training (ablation) | DreamID-Omni · IDBench-Omni R2AV (single-person ablation) | ViCLIP: 9.518 -> 14.573 | Improved |
| add progressive omni-task fine-tuning stage instead of joint multi-task training (ablation) | DreamID-Omni · IDBench-Omni R2AV (single-person ablation) | ID-Sim: 0.638 -> 0.674 | Improved |
Later papers that compared with this method
1 later accepted paper reports DreamID-Omni as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Native Audio-Visual Alignment for Generation (NeurIPS 2026 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation (NeurIPS 2025 · similarity 0.87)
- AudCast: Audio-Driven Human Video Generation by Cascaded Diffusion Transformers (CVPR 2025 · similarity 0.87)
- PrismAudio: Decomposed Chain-of-Thought and Multi-dimensional Rewards for Video-to-Audio Generation (ICLR 2026 · similarity 0.87)
- MMDisCo: Multi-Modal Discriminator-Guided Cooperative Diffusion for Joint Audio and Video Generation (ICLR 2025 · similarity 0.87)
- FaceChain-ImagineID: Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled Audio (CVPR 2024 · similarity 0.87)
- Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
