Research map / 3D vision / 3D generation
DexFuncGrasp: A Robotic Dexterous Functional Grasp Dataset Constructed from a Cost-Effective Real-Simulation Annotation System
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a cost-effective real-simulation annotation system that maps human hand poses (from RGB cameras) to a ShadowHand in IsaacGym to collect functional grasps, producing the DexFuncGrasp dataset (559 objects, ~14k grasps) and a CVAE-based functional grasp synthesis model.
Problem
Existing dexterous grasp datasets target stable grasps and lack human-like functional grasp poses (e.g., grasping a cup handle or pressing a flashlight button); prior functional grasp annotation is labor-intensive and costly.
Approach
Use human-hand-driven teleoperation with consumer RGB cameras (MediaPipe keypoints, TeachNet retargeting) to control a simulated ShadowHand in real time, verify grasps in IsaacGym, and expand annotations to other instances of the same category via Tink-based annotation transfer.
Key results
- Functional grasp synthesis model achieves 67.38% average success rate and 72.13% coverage rate across 12 categories in IsaacGym simulation (test set of 67 objects).
- Simulation SR exceeds 90% for flashlight (91.03%) and 85.77% for stapler; mug and teapot have high CR (95.98%, 98.43%) but low SR.
- Reported dataset of 559 objects across 12 categories with over 14k grasp poses, expanded from under 2000 via annotation transfer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the DFG-trained Functional Grasp Synthesis Model instead of FuncGrasp | Functional grasp generation · DFG test set, total | SR: 50.0% -> 67.38% | Helped |
| Use the DFG-trained Functional Grasp Synthesis Model instead of FuncGrasp | Functional grasp generation · DFG test set, categories with reported FuncGrasp comparison | SR: Lightbulb: 18.75% -> 74.23%; Bottle: 0.0% -> 68.62%; Flashlight: 31.25% -> 91.03%; Screwdriver: 50.0% -> 75.60%; Stapler | Mixed |
Surprising findings
- Mug and teapot achieve high coverage rates (>95%) yet low success rates because fingers knock the object down when inserting into the handle.
Limitations
- Precisely grasping cup handles remains challenging
- Only two instances per category annotated by humans; the rest rely on transfer method
- Mug and teapot grasps often fail due to thin handles and narrow space
Experimental setup
Datasets: DexFuncGrasp (DFG) DexGraspNet
Benchmarks: DFG test split (IsaacGym simulation) real-world UR10 + ShadowHand grasp experiments
Metrics: success rate coverage rate
Baselines: DexGraspNet FuncGrasp
Builds on: TeachNet Tink PointNet++ VRCNet MediaPipe IsaacGym
Closest papers by meaning
- DexGrasp Anything: Towards Universal Robotic Dexterous Grasping with Physics Awareness (CVPR 2025 · similarity 0.90)
- Grasp as You Say: Language-guided Dexterous Grasp Generation (NeurIPS 2024 · similarity 0.87)
- Scaffolding Dexterous Manipulation with Vision-Language Models (NeurIPS 2025 · similarity 0.87)
- ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR 2025 · similarity 0.86)
- Cross-Embodiment Dexterous Grasping with Reinforcement Learning (ICLR 2025 · similarity 0.86)
- Contact Map Transfer with Conditional Diffusion Model for Generalizable Dexterous Grasp Generation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.