Research map / 3D vision / Human pose estimation
Single-View Scene Point Cloud Human Grasp Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes S2HGrasp, an end-to-end framework that generates human grasps (MANO parameters) from single-view scene point clouds using a Global Perception module and a DiffuGrasp diffusion decoder, and introduces the S2HGD synthetic dataset.
Problem
Generating human grasps from single-view scene point clouds, where object point clouds are incomplete and contain many scene (tabletop) points, causing hand penetration into unseen object parts and collisions with the scene.
Approach
Combines multi-task global shape completion and classification on scene features (used only in training) with a diffusion model that denoises normalized MANO hand parameters conditioned on scene features, adding a hand encoder that embeds the MANO mesh and a plane loss to avoid tabletop collisions.
Key results
- On View-S2HGD, S2HGrasp reduces penetration volume to 6.58 cm3 from 13.54 cm3 for GraspCVAE, with grasp displacement 2.73 cm vs 3.10 cm.
- Versus two-stage PCN+DG on Object-S2HGD, S2HGrasp achieves penetration volume 5.58 cm3 vs 7.68 and contact ratio 98.67% vs 72.85%.
- Global Perception module lowers penetration volume from 11.54 to 6.58 cm3 when added to DiffuGrasp on View-S2HGD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use S2HGrasp instead of GraspCVAE | human grasp generation · View-S2HGD | penetration volume (cm3): 13.54 -> 6.58 | Helped |
| Use S2HGrasp instead of GraspCVAE | human grasp generation · Object-S2HGD | penetration volume (cm3): 14.48 -> 5.58 | Helped |
| Use S2HGrasp instead of GraspCVAE | human grasp generation · HO3D-object | penetration volume (cm3): 4.87 -> 3.30 | Helped |
| Use S2HGrasp instead of GraspCVAE | human grasp generation · Obman-object | penetration volume (cm3): 4.62 -> 4.62 | No clear effect |
| Add the Global Perception module (ablation) | DiffuGrasp on View-S2HGD · View-S2HGD | penetration volume (cm3): 11.54 -> 6.58 | Helped |
| Replace CVAE with DiffuGrasp (ablation) | grasp generation on View-S2HGD · View-S2HGD | penetration volume (cm3): 13.54 -> 11.54 | Helped |
| Replace direct hand-parameter decoding with a hand encoder (ablation) | S2HGrasp on View-S2HGD · View-S2HGD | grasp displacement mean (cm): 3.13 -> 2.73 | Helped |
| Use a two-stage PCN completion plus GraspCVAE pipeline (ablation) | grasp generation on Object-S2HGD · Object-S2HGD | penetration volume (cm3): 5.58 -> 11.34 | Hurt |
| Use a two-stage PCN completion plus DiffuGrasp pipeline (ablation) | grasp generation on Object-S2HGD · Object-S2HGD | penetration volume (cm3): 5.58 -> 7.68 | Hurt |
| Use S2HGrasp instead of GraspCVAE | human grasp generation · View-S2HGD | grasp displacement mean (cm): 3.10 -> 2.73 | Helped |
| Use S2HGrasp instead of GraspCVAE | human grasp generation · Object-S2HGD | grasp displacement mean (cm): 3.69 -> 3.26 | Helped |
Surprising findings
- Penetration of generated grasps on S2HGD is even lower than the ground-truth grasps in some settings (e.g., 0.21 cm vs 0.27 cm depth).
- GraspCVAE shows smaller grasp displacement partly because its larger penetration volume produces stronger finger forces in simulation, so displacement alone is misleading.
Limitations
- Fails on complex object shapes where global shape is hard to infer from partial point clouds.
- Fails when only a small part of the object is visible, giving too few points for a reasonable grasp.
- Dataset is synthetic (BlenderProc simulation) with a single table setting and single object per scene.
Experimental setup
Datasets: S2HGD OakInk Obman HO3D ShapeNet
Benchmarks: S2HGD (View-S2HGD, Object-S2HGD) HO3D-object Obman-object
Metrics: penetration depth penetration volume grasp displacement perceptual score contact ratio diversity (variance of rotation axes and angles)
Baselines: GraspCVAE (GraspTTA) PCN+GC PCN+DG SceneDiffuser (no hand encoder)
Builds on: GraspTTA DDPM DDIM PCN MANO
Closest papers by meaning
- LatentHOI: On the Generalizable Hand Object Motion Generation with Latent Hand Diffusion. (CVPR 2025 · similarity 0.86)
- DiffGrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model (AAAI 2025 · similarity 0.86)
- MANUS: Markerless Grasp Capture using Articulated 3D Gaussians (CVPR 2024 · similarity 0.85)
- CADGrasp: Learning Contact and Collision Aware General Dexterous Grasping in Cluttered Scenes (NeurIPS 2025 · similarity 0.85)
- RAGG: Retrieval-Augmented Grasp Generation Model (AAAI 2025 · similarity 0.85)
- Grasp2Grasp: Vision-Based Dexterous Grasp Translation via Schrödinger Bridges (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.