Research map / 3D vision / 3D generation
G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a category-conditioned 3D diffusion prior over joint hand-object interaction grids (latent object SDF plus skeletal distance field for the hand) and uses score distillation to guide video reconstruction and human grasp synthesis.
Problem
Hand-object interactions are typically modeled conditionally or with heterogeneous representations; there is no generative model that jointly generates 3D object shape and hand configuration, and task-specific methods for HOI reconstruction and grasp synthesis lack a shared prior.
Approach
Represent the hand as a skeletal distance field (15 joint-distance channels) so it is spatially aligned with a latent object SDF, forming a homogeneous interaction grid in a hand-centric frame; train a 3D UNet diffusion model on it and use SDS-style score distillation to guide test-time optimization for downstream tasks.
Key results
- On HOI4D reconstruction, G-HOP reaches F@5mm 0.76 and F@10mm 0.97 for object error, versus 0.62 and 0.91 for DiffHOI, and CDh 18.4 versus 48.7.
- Test-time optimization for reconstruction takes about an hour per clip (15000 iterations), reported as 85% faster than DiffHOI.
- In user studies on HO3D and 3DW, G-HOP grasps were preferred over GraspTTA and ground-truth annotations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the 3D G-HOP prior instead of DiffHOI’s 2D prior | HOI reconstruction · HOI4D | F5: 0.62 -> 0.76 | Helped |
| Use the 3D G-HOP prior instead of DiffHOI’s 2D prior | HOI reconstruction · HOI4D | CD: 0.8 -> 0.4 | Helped |
| Jointly model hand and object rather than condition on hand pose (ablation) | G-HOP HOI reconstruction · HOI4D | MPJPE: 1.14 -> 1.05 | Helped |
| Jointly model hand and object rather than condition on hand pose (ablation) | G-HOP HOI reconstruction · HOI4D | F5: 0.66 -> 0.76 | Helped |
| Use G-HOP rather than GraspTTA | human grasp synthesis · HO3D | Disp.: 2.89 -> 0.95 | Helped |
| Use G-HOP rather than GraspTTA | human grasp synthesis · 3DW | contact area: 0.18 -> 0.25 | Helped |
| Use G-HOP rather than GraspTTA | human grasp synthesis · HO3D | vol: 5.25 -> 11.46 | Hurt |
| Use G-HOP rather than GraspTTA | human grasp synthesis · 3DW | Disp.: 1.54 -> 1.85 | Mixed |
| Rank generated grasps using the proposed grasp score | G-HOP grasp synthesis · HO3D | Disp.: 0.95 -> 0.71 | Helped |
Surprising findings
- Diffusing the MANO pose parameters jointly with object latents worked poorly compared with the skeletal distance field representation.
- A joint HOI prior modeled in 2D image space performed even worse than DiffHOI.
Limitations
- Requires object category as input, which may limit scaling.
- No explicit mechanism guarantees hand-object contact.
- Limited training data; model is not at the scale of generative models in other domains.
Experimental setup
Datasets: HOI4D HO3D 3D Warehouse (3DW) ObMan DexYCB ContactPose YCB
Benchmarks: HOI4D HO3D 3DW
Metrics: F-score@5mm F-score@10mm Chamfer distance CDh MPJPE AUC mean intersection volume max/avg depth simulation displacement contact ratio
Baselines: iHOI HHOR DiffHOI GraspTTA G-HOP (Cond) Grasping Fields GT
Builds on: DiffHOI score distillation sampling VQ-VAE MANO CLIP
Closest papers by meaning
- LatentHOI: On the Generalizable Hand Object Motion Generation with Latent Hand Diffusion. (CVPR 2025 · similarity 0.90)
- InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusion (CVPR 2024 · similarity 0.89)
- DiffGrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model (AAAI 2025 · similarity 0.88)
- HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactions (CVPR 2024 · similarity 0.88)
- Single-view Image to Novel-view Generation for Hand-Object Interactions (AAAI 2025 · similarity 0.88)
- GEARS: Local Geometry-aware Hand-object Interaction Synthesis (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.