Research map / 3D vision / 3D generation
GenH2R: Learning Generalizable Human-to-Robot Handover via Scalable Simulation Demonstration and Imitation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a large synthetic human-to-robot handover simulator (GenH2R-Sim), generates distillation-friendly demonstrations via landmark planning, and trains a forecast-aided 4D point-cloud imitation policy that outperforms HandoverSim2real in simulation and on a real robot.
Problem
Learning generalizable vision-based human-to-robot handover policies that handle unseen object geometry and complex human trajectories, which is limited by scarce simulated human handover assets and demonstrations.
Approach
Procedurally scale handover scenes (ShapeNet objects, DexGraspNet grasps, Bezier-curve hand trajectories), generate demonstrations by landmark planning that replans on forecastable future object poses to preserve vision-action correlation, and distill them with a PointNet++ policy using ICP-based flow features and a future object-motion forecasting auxiliary loss.
Key results
- Landmark-planning policy trained on t0 improves success rate over Handover-Sim2real trained on s0 by 11.34%, 16.90%, 12.26%, and 15.93% across the four test sets.
- Removing the flow input drops success rate from 41.43% to 31.66% on t0 t1 ablation; removing the prediction loss drops it to 39.18%.
- Real-world user study: 90% success in simple setting and 70% in complex setting versus 56.7% and 33.3% for Handover-Sim2real.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with GenH2R-Sim data instead of the small-scale s0 training set | Landmark Planning · s0 (Sequential) | success rate: +8.79% | Helped |
| train with GenH2R-Sim data instead of the small-scale s0 training set | Landmark Planning · s0 (Simultaneous) | success rate: +6.48% | Helped |
| train with GenH2R-Sim data instead of the small-scale s0 training set | Landmark Planning · t0 | success rate: +11.80% | Helped |
| train with GenH2R-Sim data instead of the small-scale s0 training set | Landmark Planning · t1 | success rate: +14.13% | Helped |
| use landmark planning instead of dense planning | GenH2R imitation policy trained on t0 · t1 | average success: +9.5% | Helped |
| use destination planning instead of landmark planning | GenH2R imitation policy trained on t0 · s0 | success rate: 73.38% / 69.68% decrease | Hurt |
| use 10% of the GenH2R-Sim training data | GenH2R imitation learning · t1 | success rate: 5.93% drop | Hurt |
| add flow information to the input (ablation) | Forecast-aided 4D imitation learning · t0 | success rate: 31.66 -> 41.43; 9.77% decrease without flow | Helped |
| add prediction loss for future object motion (ablation) | Forecast-aided 4D imitation learning · t0 | success rate: 39.18 -> 41.43; 2.25% decrease without prediction | Helped |
| add both flow information and prediction loss (ablation) | Forecast-aided 4D imitation learning · t0 | success rate: 37.04 -> 41.43; 4.39% decrease without both components | Helped |
Surprising findings
- Destination planning demonstrations trained on t0 yield near-zero success (0.93% on s0 sequential), showing that foresighted demonstrations harm distillation under complex trajectories.
Limitations
- Synthetic human hand motion is not guaranteed to mimic real human behavior.
- Real-world evaluation uses only 6 users and 5 objects.
- Reproduced baseline numbers differ notably from originally reported numbers.
Experimental setup
Datasets: GenH2R-Sim HandoverSim s0 (DexYCB-based) ShapeNet DexGraspNet HOI4D
Benchmarks: HandoverSim s0 (Sequential) HandoverSim s0 (Simultaneous) GenH2R-Sim t0 GenH2R-Sim t1
Metrics: success rate execution time average success (AS)
Baselines: Handover-Sim2real GA-DDPG OMG Planner Destination Planning Dense Planning
Builds on: HandoverSim DexGraspNet OMG Planner PointNet++ Handover-Sim2real
Closest papers by meaning
- MobileH2R: Learning Generalizable Human to Mobile Robot Handover Exclusively from Scalable and Diverse Synthetic Data (CVPR 2025 · similarity 0.95)
- ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR 2025 · similarity 0.85)
- GENMANIP: LLM-driven Simulation for Generalizable Instruction-Following Manipulation (CVPR 2025 · similarity 0.85)
- Scaffolding Dexterous Manipulation with Vision-Language Models (NeurIPS 2025 · similarity 0.85)
- Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation (ICLR 2025 · similarity 0.85)
- ManiSkill-HAB: A Benchmark for Low-Level Manipulation in Home Rearrangement Tasks (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.