Research map / 3D vision / 3D generation
DiffGrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffGrasp, a single conditional diffusion transformer that generates whole-body SMPL-X grasping sequences conditioned on object shape (BPS) and object motion, trained with contact-aware losses and inference-time guidance.
Problem
Generating realistic whole-body human motion sequences with fine finger-object grasping conditioned on a 3D object motion sequence, where prior methods either lack hand pose detail or only model static or single-frame grasps.
Approach
A single conditional diffusion transformer jointly predicts SMPL-X whole-body pose and wrist-to-object translations, trained with contact-aware reconstruction and interaction losses weighted by contact labels, and at inference uses reconstruction guidance (grasp stabilization, hand-object contact, feet penetration) to refine samples.
Key results
- On GRAB, DiffGrasp reaches F1 0.7840 and hands JPE 20.99 cm versus OMOMO-V3's 0.1028 F1 and 32.72 cm.
- On ARCTIC, DiffGrasp achieves F1 0.8067 and MPVPE 8.00 cm, beating all OMOMO variants.
- On unseen GRAB objects, DiffGrasp attains F1 0.9233 versus 0.2923 for OMOMO-V3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DiffGrasp instead of OMOMO-V3 | Whole-body object grasp synthesis · GRAB | F1: 0.1028 -> 0.7840 | Helped |
| Use DiffGrasp instead of OMOMO-V3 | Whole-body object grasp synthesis · ARCTIC | F1: 0.3338 -> 0.8067 | Helped |
| Add contact-aware reconstruction loss (ablation) | DiffGrasp without interaction loss · GRAB | F1: 0.3861 -> 0.5319 | Helped |
| Replace simple reconstruction loss with contact-aware reconstruction loss (ablation) | DiffGrasp without interaction loss · GRAB | F1: 0.3961 -> 0.5319 | Helped |
| Add contact-aware interaction loss (ablation) | DiffGrasp without reconstruction loss · GRAB | F1: 0.5543 -> 0.6982 | Helped |
| Add contact-aware reconstruction loss (ablation) | DiffGrasp without interaction loss · GRAB | F1: 0.5319 -> 0.6982 | Helped |
| Add part-wise positional encoding PE^P (ablation) | DiffGrasp · GRAB | F1: 0.6448 -> 0.6982 | Helped |
| Add inference guidance (ablation) | DiffGrasp · GRAB | F1: 0.6982 -> 0.7840 | Helped |
| Add inference guidance (ablation) | DiffGrasp · GRAB | Collision percentage: 0.0033 -> 0.0023 | Hurt |
Surprising findings
- Adding the contact-aware interaction loss gives the highest F1 among loss ablations but yields less natural hand poses, motivating the combined loss.
- OMOMO and OMOMO-V3 achieve lower foot sliding (FS) than DiffGrasp, which the authors attribute to static generated body poses.
Limitations
- May generate self-penetration of the human body mesh due to no self-penetration constraint.
- Generated grasp poses can be unrealistic because no physical hand-object constraints are imposed.
- Walking during grasping is not achieved due to dataset limitations.
Experimental setup
Datasets: GRAB ARCTIC
Benchmarks: GRAB ARCTIC
Metrics: Hands JPE MPJPE MPVPE foot sliding collision percentage collision depth F1 contact score contact distance human perceptual score
Baselines: OMOMO OMOMO-V2 OMOMO-V3 IMoS COOP
Builds on: OMOMO DDPM InterDiff SMPL-X BPS
Closest papers by meaning
- CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objects (NeurIPS 2025 · similarity 0.91)
- HandDiffuse: Generative Controllers for Two-Hand Interactions via Diffusion Models (AAAI 2025 · similarity 0.89)
- HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data (CVPR 2024 · similarity 0.89)
- LatentHOI: On the Generalizable Hand Object Motion Generation with Latent Hand Diffusion. (CVPR 2025 · similarity 0.89)
- Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modeling (AAAI 2024 · similarity 0.89)
- G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.