Research map / Detection and segmentation / Open-set and few-shot detection
Guiding Human-Object Interactions with Rich Geometry and Relations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ROG, a diffusion framework for text-driven human-object interaction synthesis that represents objects with boundary and Poisson-disk key points, builds an Interactive Distance Field, and refines generated motion via a spatio-temporal relation model with L-BFGS guidance.
Problem
Text-driven human-object interaction (HOI) synthesis often uses simplified object representations (centroid or nearest point), which overlook geometric complexity and reduce interaction fidelity.
Approach
Represents the object with 24 key points (8 bounding-box-nearest boundary points plus 16 Poisson-disk samples), builds a human-joint-to-object-keypoint distance matrix (IDF) over time, adds an IDF loss to the motion diffusion model, and uses a video-diffusion-transformer-style relation model with spatial and temporal attention to refine the IDF during denoising via L-BFGS guidance.
Key results
- ROG reaches R-Precision Top-1 0.706 vs 0.630 for CHOIS on FullBodyManipulation.
- ROG improves MDev to 5.815 vs 13.408 for CHOIS (real motions 4.846) and achieves FID 5.119 vs 5.227 for CHOIS.
- Ablation: adding full distance matrix (object key points plus human joints) gives the best overall metrics over centroid-only distance (C) and no guidance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ROG instead of CHOIS | FullBodyManipulation HOI synthesis · FullBodyManipulation test | R-Precision Top-1: 0.630 -> 0.706 | Helped |
| Use ROG instead of CHOIS | FullBodyManipulation HOI synthesis · FullBodyManipulation test | FID: 5.227 -> 5.119 | Helped |
| Use ROG instead of CHOIS | FullBodyManipulation HOI synthesis · FullBodyManipulation test | Contact Percentage (C%): 0.444 -> 0.466 | Helped |
| Add object key points to the baseline (ablation) | MDM-based HOI synthesis · FullBodyManipulation test | FID: 9.972 -> 7.514 | Helped |
| Add IDF loss (ablation) | MDM-based HOI synthesis with object key points · FullBodyManipulation test | Contact Percentage (C%): 0.417 -> 0.424 | Helped |
| Add IDF loss (ablation) | MDM-based HOI synthesis with object key points · FullBodyManipulation test | Collision Percentage (Coll%): 0.205 -> 0.194 | Helped |
| Add guidance using the full distance matrix (ablation) | ROG with IDF loss · FullBodyManipulation test | FID: 5.902 -> 5.119 | Helped |
| Replace the centroid distance matrix with the full distance matrix (ablation) | ROG with IDF loss and guidance · FullBodyManipulation test | Contact Percentage (C%): 0.364 -> 0.466 | Helped |
| Replace the centroid distance matrix with the full distance matrix (ablation) | ROG with IDF loss and guidance · FullBodyManipulation test | Motion Deviation (MDev): 9.936 -> 5.815 | Helped |
Surprising findings
- The centroid-only distance variant (C) yields higher Collision Percentage and worse MDev than the full key-point matrix, and the guidance-free variant performs worse on most metrics.
Limitations
- Excluded hand and finger motions in the dataset result in low-fidelity hand details.
- Evaluated only on a single dataset (FullBodyManipulation) with articulated objects removed.
Experimental setup
Datasets: FullBodyManipulation
Benchmarks: FullBodyManipulation
Metrics: FID R-Precision Diversity Foot Sliding Contact Percentage Collision Percentage Motion Deviation
Baselines: InterGen MDM HOI-Diff CHOIS
Builds on: Motion Diffusion Model VDT video diffusion transformer CHOIS Poisson disk sampling L-BFGS guidance from Wang et al.
Closest papers by meaning
- CG-HOI: Contact-Guided 3D Human-Object Interaction Generation (CVPR 2024 · similarity 0.91)
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.89)
- Real2Sim in HOI: Toward Physically Plausible HOI Reconstruction from Monocular Videos (NeurIPS 2026 · similarity 0.88)
- HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusion (NeurIPS 2025 · similarity 0.88)
- AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animation (ICLR 2025 · similarity 0.88)
- HOIAnimator: Generating Text-prompt Human-object Animations using Novel Perceptive Diffusion Models (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.