Research map / Detection and segmentation / Open-set and few-shot detection
UniGraspTransformer: Simplified Policy Distillation for Scalable Dexterous Robotic Grasping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains per-object PPO grasping policies on 3,200 objects, generates 3.2M successful trajectories, and distills them offline into a 12-block Transformer universal dexterous grasping policy that beats UniDexGrasp++.
Problem
Universal dexterous grasping policies degrade when one network must handle thousands of diverse objects, and prior pipelines (UniDexGrasp++) are complex multi-step training processes.
Approach
Replace multi-stage online distillation with a simple pipeline: train dedicated RL teachers per object, generate large offline trajectory sets, and supervised-distill them into a large self-attention Transformer that scales with depth and handles state-based and vision-based inputs.
Key results
- State-based: 91.2% success on seen objects vs 87.9% for UniDexGrasp++; vision-based: 88.9% vs 85.4% (+3.5%).
- Vision-based unseen-category success 86.8% vs 76.7% for UniDexGrasp++ (+10.1%).
- Self-attention depth ablation: 85.5% (0 blocks), 89.7% (6 blocks), 91.2% (12 blocks); offline distillation 92.5% vs DAgger 88.2% on 400 objects.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace UniDexGrasp++ with UniGraspTransformer | Universal dexterous grasping · State-based, seen objects | success rate: 87.9 -> 91.2 | Helped |
| replace UniDexGrasp++ with UniGraspTransformer | Universal dexterous grasping · State-based, unseen objects from seen categories | success rate: 84.3 -> 89.2 | Helped |
| replace UniDexGrasp++ with UniGraspTransformer | Universal dexterous grasping · State-based, unseen objects from unseen categories | success rate: 83.1 -> 88.3 | Helped |
| replace UniDexGrasp++ with UniGraspTransformer | Universal dexterous grasping · Vision-based, seen objects | success rate: 85.4 -> 88.9 | Helped |
| replace UniDexGrasp++ with UniGraspTransformer | Universal dexterous grasping · Vision-based, unseen objects from seen categories | success rate: 79.6 -> 87.3 | Helped |
| replace UniDexGrasp++ with UniGraspTransformer | Universal dexterous grasping · Vision-based, unseen objects from unseen categories | success rate: 76.7 -> 86.8 | Helped |
| increase trajectories per object from 0.2K to 1K (ablation) | UniGraspTransformer · State-based, seen objects | success rate: 87.2 -> 91.2 | Helped |
| increase self-attention blocks from 0 to 12 (ablation) | UniGraspTransformer · State-based, seen objects | success rate: 85.5 -> 91.2 | Helped |
| increase distilled object policies from 400 to 3,200 (ablation) | UniGraspTransformer · Corresponding seen-object set | success rate: 92.5 -> 91.2 | Hurt |
| use offline UniGraspTransformer distillation instead of online DAgger (ablation) | Universal policy distilled from 400 dedicated policies · Corresponding seen-object set | success rate: 88.2 -> 92.5 | Helped |
| add object features to proprioception, previous action, and object state inputs (ablation) | UniGraspTransformer · State-based, seen objects | success rate: 78.4 -> 86.6 | Helped |
| add partial-cloud center and PCA rotation estimates (ablation) | UniGraspTransformer · Vision-based, seen objects | success rate: 83.2 -> 88.9 | Helped |
Surprising findings
- Distilling 400 policies gives higher success (92.5%) than distilling all 3,200 (91.2%), with a slight decline as object count grows.
Limitations
- Relies on per-object RL teachers, so training cost scales with the number of objects (80 hours for 3,200 policies)
- Performance drops when moving from state-based to vision-based inputs and from seen to unseen objects
Experimental setup
Datasets: UniDexGrasp++ dexterous grasping dataset (3,200 objects, 133 categories)
Benchmarks: UniDexGrasp++ dexterous grasping dataset (seen objects, unseen objects in seen categories, unseen categories)
Metrics: success rate
Baselines: PPO DAPG ILAD GSL UniDexGrasp UniDexGrasp++ DAgger
Builds on: UniDexGrasp++ PPO DAgger PointNet
Closest papers by meaning
- ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR 2025 · similarity 0.89)
- Efficient Residual Learning with Mixture-of-Experts for Universal Dexterous Grasping (ICLR 2025 · similarity 0.89)
- DexGrasp Anything: Towards Universal Robotic Dexterous Grasping with Physics Awareness (CVPR 2025 · similarity 0.88)
- Cross-Embodiment Dexterous Grasping with Reinforcement Learning (ICLR 2025 · similarity 0.88)
- Scaffolding Dexterous Manipulation with Vision-Language Models (NeurIPS 2025 · similarity 0.88)
- Dexterous Grasp Transformer (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.