Research map / 3D vision / 3D generation
Open-world Hand-Object Interaction Video Generation Based on Structure and Contact-aware Representation
Haodong Yan, Hang Yu, Zhide Zhong, Weilin Yuan, Xin Gong, Zehang Luo, Chengxi Heyu, Junfeng Li, Wenxuan Song, Shunbo Zhou, Haoang Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the structure and contact-aware representation with optical flow (ablation) | SCAR_C · Taco | Subject consistency (SC): 0.916 -> 0.889 | Hurt |
| replace the representation with hand-object masks only (ablation) | SCAR_C · Taco | Subject consistency (SC): 0.916 -> 0.903 | Hurt |
| replace the representation with depth maps only (ablation) | SCAR_C · Taco | Subject consistency (SC): 0.916 -> 0.889 | Hurt |
| add hand-object contours (ablation) | SCAR_C · Taco | Subject consistency (SC): 0.899 -> 0.916 | Improved |
| add contact region estimation (ablation) | SCAR_C · Taco | ViCLIP score (VCS): 0.179 -> 0.187 | Improved |
| add depth maps (ablation) | SCAR_C · Taco | Subject consistency (SC): 0.901 -> 0.916 | Improved |
| add rendered 2D hand keypoints to the representation (ablation) | SCAR_C · Taco | Subject consistency (SC): 0.916 -> 0.891 | Hurt |
| joint generation of representation and video instead of two-stage optical flow pipeline (FLOVD) | CogVideoX I2V-5B · Taco | VBench Total Score: 8.619 -> 8.793 | Improved |
| fine-tune with SCAR joint generation on CogVideoX | CogVideoX I2V-5B · Taste-Rob | VBench Total Score: 8.959 -> 9.043 | Improved |
| fine-tune with SCAR joint generation on Wan2.1 | Wan2.1 I2V-14B · Taco | VBench Total Score: 8.792 -> 8.899 | Improved |
| fine-tune with SCAR on CogVideoX | CogVideoX I2V-5B · Taste-Rob | Imaging quality (IQ): 0.688 -> 0.696 | No clear effect |
Closest papers by meaning
- SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios (NeurIPS 2025 · similarity 0.92)
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.91)
- ManiVideo: Generating Hand-Object Manipulation Video with Dexterous and Generalizable Grasping (CVPR 2025 · similarity 0.89)
- Single-view Image to Novel-view Generation for Hand-Object Interactions (AAAI 2025 · similarity 0.89)
- Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model (CVPR 2025 · similarity 0.89)
- Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
