Research map / 3D vision / 3D generation
CG-HOI: Contact-Guided 3D Human-Object Interaction Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates dynamic 3D human-object interaction sequences from text by jointly diffusing human SMPL motion, object pose, and body-surface contact distances with cross-attention, plus contact-based guidance at inference.
Problem
Generating realistic, physically plausible 3D human-object interaction sequences from text, where human and object motion are interdependent and existing methods either model humans alone or objects in static scenes.
Approach
Jointly diffuse human body (SMPL), object transformation, and contact distances from 128 body surface markers to the object, linked by human-object-contact cross-attention; weight object hypotheses per body marker by predicted contact; and guide sampling so recomputed contact matches predicted contact.
Key results
- On CHAIRS text-conditioned HOI, the method reaches R-Precision 0.74 and FID 6.45 vs. InterDiff 0.69 / 7.53 and MDM 0.53 / 9.23.
- On BEHAVE text-conditioned HOI, R-Precision 0.62 and FID 6.31 versus InterDiff 0.53 / 8.70 and MDM 0.49 / 9.21.
- A 32-participant user study significantly favored the method over baselines for realism and text coherence.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add human-object-contact cross-attention (ablation) | CG-HOI · BEHAVE and CHAIRS | R-Prec. (top-3): 0.35 -> 0.62; 0.49 -> 0.74 | Helped |
| add joint contact prediction (ablation) | CG-HOI · BEHAVE and CHAIRS | R-Prec. (top-3): 0.41 -> 0.62; 0.41 -> 0.74 | Helped |
| use joint rather than separately trained contact prediction (ablation) | CG-HOI · BEHAVE and CHAIRS | R-Prec. (top-3): 0.47 -> 0.62; 0.52 -> 0.74 | Helped |
| add contact-based object transformation weighting (ablation) | CG-HOI · BEHAVE and CHAIRS | R-Prec. (top-3): 0.55 -> 0.62; 0.64 -> 0.74 | Helped |
| add contact-based inference guidance (ablation) | CG-HOI · BEHAVE and CHAIRS | R-Prec. (top-3): 0.59 -> 0.62; 0.70 -> 0.74 | Helped |
| replace InterDiff with CG-HOI for text-conditioned HOI generation | Text-Cond. HOI · BEHAVE and CHAIRS | R-Prec. (top-3): 0.53 -> 0.62; 0.69 -> 0.74 | Helped |
Limitations
- Handles only a single object at a time; multi-object interactions are not modeled.
- Requires expensive 3D HOI motion capture for training; depends on manual text annotations.
- Generated sequences are short (32 frames on CHAIRS, 64 on BEHAVE, up to 3 seconds).
Experimental setup
Datasets: BEHAVE CHAIRS ScanNet++
Benchmarks: BEHAVE CHAIRS
Metrics: R-Precision (top-3) FID Diversity MultiModality user study
Baselines: MDM InterDiff
Builds on: DDPM MDM InterDiff classifier-free guidance CLIP
Closest papers by meaning
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.93)
- InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction (NeurIPS 2024 · similarity 0.92)
- InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing (CVPR 2025 · similarity 0.91)
- HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusion (NeurIPS 2025 · similarity 0.91)
- Guiding Human-Object Interactions with Rich Geometry and Relations (CVPR 2025 · similarity 0.91)
- Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction (CVPR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.