Research map / 3D vision / Human pose estimation
Dexterous Grasp Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DGTR, a transformer-decoder set-prediction model that generates diverse, high-quality dexterous grasps from an object point cloud in one forward pass, using dynamic-static matching training and adversarial-balanced test-time adaptation.
Problem
Dexterous grasp generation from complete object point clouds where generative and vanilla discriminative models yield either low diversity or require multiple inference passes with input rotations; set-prediction training also suffers from Hungarian matching instability when penetration loss is appl
Approach
Formulates dexterous grasp generation as set prediction with learnable grasp queries in a transformer decoder over PointNet++ features, trains with a dynamic-then-static Hungarian matching schedule to avoid model collapse under penetration loss, and refines grasps at test time with a pair of adversarial penetration/contact losses that update only the hand pose.
Key results
- DGTR reaches Q1 of 0.0515 with 75.78% non-penetration and 41.0% grasp success in one forward pass, with diversity δt 47.77 and δr 51.66 on DexGraspNet.
- DGTR* (top-4 selection) achieves Q1 0.0921 and 99.51% non-penetration, comparable to DDG's quality reference.
- DSMT raises Q1 from 0.0115 to 0.0278 and cuts maximal penetration from 0.869 to 0.466 cm; AB-TTA raises Q1 to 0.0515.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DGTR instead of UniDexGrasp in one forward pass | Dexterous grasp generation · DexGraspNet | δt: 9.64 -> 47.77 | Helped |
| Use DGTR instead of UniDexGrasp in one forward pass | Dexterous grasp generation · DexGraspNet | Q1: 0.0462 -> 0.0515 | Helped |
| Select top-4 grasps by contact points and object penetration | DGTR · DexGraspNet | Q1: 0.0515 -> 0.0921 | Helped |
| Add AB-TTA (ablation) | DGTR · DexGraspNet | Q1: 0.0278 -> 0.0515 | Helped |
| Add DSMT (ablation) | DGTR · DexGraspNet | Q1: 0.0115 -> 0.0515 | Helped |
| Add DSMT (ablation) | DGTR · DexGraspNet | Pen.: 0.869 -> 0.421 | Helped |
| Add static matching and SMPT after DMT (ablation) | DGTR · DexGraspNet | Pen.: 0.869 -> 0.466 | Helped |
| Omit SMPT after DMT + SMW (ablation) | DGTR · DexGraspNet | Pen.: 0.466 -> 0.879 | Hurt |
| Use 4 grasp queries instead of 16 (ablation) | DGTR · DexGraspNet | Q1: 0.0278 -> 0.0392 | Helped |
| Use 4 grasp queries instead of 16 (ablation) | DGTR · DexGraspNet | δt: 47.77 -> 18.40 | Hurt |
| Use 32 grasp queries instead of 16 (ablation) | DGTR · DexGraspNet | δt: 47.77 -> 72.13 | Helped |
| Set penetration loss weight to 500 (ablation) | DGTR · DexGraspNet | Q1: 0.0515 -> 0.0020 | Hurt |
Surprising findings
- Large object penetration loss weight (λpen=500) causes model collapse to nearly identical grasps, and gradually warming up the penetration weight does not fix this.
- Single-pass DGTR surpasses UniDexGrasp with 16 forward passes on translation and rotation diversity.
Limitations
- Quality decreases as the number of queries increases (Q1 falls from 0.0392 at N=4 to 0.0170 at N=64).
- Evaluated only on DexGraspNet with the ShadowHand; generalization to other hands or real-world settings is not shown.
- Vanilla TTA with penetration and distance loss has limited effect, so the test-time refinement requires specifically designed losses.
Experimental setup
Datasets: DexGraspNet
Benchmarks: DexGraspNet
Metrics: Mean Q1 non-penetration ratio torque balance ratio grasping success rate maximal penetration depth occupancy proportion of translations occupancy proportion of rotations occupancy proportion of joint angles
Baselines: GraspTTA UniDexGrasp SceneDiffuser DDG ContactNet-TTA
Builds on: DETR PointNet++ UniDexGrasp DexGraspNet
Closest papers by meaning
- UniGraspTransformer: Simplified Policy Distillation for Scalable Dexterous Robotic Grasping (CVPR 2025 · similarity 0.87)
- DexGrasp Anything: Towards Universal Robotic Dexterous Grasping with Physics Awareness (CVPR 2025 · similarity 0.86)
- Contact Map Transfer with Conditional Diffusion Model for Generalizable Dexterous Grasp Generation (NeurIPS 2025 · similarity 0.85)
- Grasp as You Say: Language-guided Dexterous Grasp Generation (NeurIPS 2024 · similarity 0.85)
- Generalizing 6-DoF Grasp Detection via Domain Prior Knowledge (CVPR 2024 · similarity 0.85)
- CADGrasp: Learning Contact and Collision Aware General Dexterous Grasping in Cluttered Scenes (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.