Research map / Video / Video generation
OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation
Donghao Zhou, Guisheng Liu, Hao Yang, Jiatong Li, Jingyu Lin, Xiaohu Huang, Yichen Liu, Xin Gao, Cunjian Chen, Shilei Wen, Chi-Wing Fu, Pheng-Ann Heng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add unified channel-wise conditioning instead of token concatenation (ablation) | OmniShow · Ablation (a) R2V | FaceSim: 0.601 -> 0.707 | Improved |
| add reference reconstruction loss (ablation) | OmniShow · Ablation (a) R2V | FaceSim: 0.678 -> 0.707 | Improved |
| add attention-map constraints in gated local-context attention (ablation) | OmniShow · Ablation (b) RA2V | Sync-C: 2.201 -> 9.023 | Improved |
| add audio context (local window) (ablation) | OmniShow · Ablation (b) RA2V | Sync-D: 7.878 -> 7.419 | Improved |
| add adaptive gating (ablation) | OmniShow · Ablation (b) RA2V | Sync-D: 7.819 -> 7.419 | Improved |
| add attention-map constraints (ablation) | OmniShow · Ablation (b) RA2V | AES: 0.545 -> 0.540 | Hurt |
| use decoupled-then-joint training instead of single-stage RA2V (ablation) | OmniShow · Ablation (c) RA2V | NexusScore: 0.345 -> 0.364 | Improved |
| use decoupled-then-joint training instead of R2V->RA2V multi-stage (ablation) | OmniShow · Ablation (c) RA2V | Sync-D: 13.23 -> 8.14 | Improved |
| pretrain on A2V then RA2V instead of ours (ablation) | OmniShow · Ablation (c) RA2V | Sync-D: 7.38 vs 8.14 | No clear effect |
| use native RoPE instead of spatiotemporal shift for pseudo-frames (ablation) | OmniShow · R2V | FaceSim: 0.279 -> 0.707 | Improved |
| use native RoPE instead of temporal shift (ablation) | OmniShow · R2V | FaceSim: 0.675 -> 0.707 | Improved |
| use context window 5 instead of 11 (ablation) | OmniShow · RA2V | Sync-C: 7.020 -> 9.023 | Improved |
Closest papers by meaning
- HOIGen-1M: A Large-scale Dataset for Human-Object Interaction Video Generation (CVPR 2025 · similarity 0.89)
- Semantic-Aware Human Object Interaction Image Generation (ICML 2024 · similarity 0.87)
- VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis (CVPR 2025 · similarity 0.86)
- OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation (CVPR 2025 · similarity 0.86)
- InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction (NeurIPS 2024 · similarity 0.85)
- Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
