Research map / Large language models / Agents, code and math
MobileH2R: Learning Generalizable Human to Mobile Robot Handover Exclusively from Scalable and Diverse Synthetic Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MobileH2R, a framework that learns vision-based human-to-mobile-robot handover entirely from scalable synthetic full-body human motion, safe imitation-friendly CHOMP demonstrations, and a 4D point-cloud imitation policy producing coordinated base-arm actions.
Problem
Generalizable vision-based handover of objects from humans to a mobile robot in a large workspace, without real-world demonstrations, which existing fixed-base H2R methods and mocap-limited datasets do not address.
Approach
Combines GMD-generated full-body human motion, an interactive human agent, and a CHOMP planner with future obstacle avoidance, final-pose constraints, and a learned vision-state recovery estimator to produce imitation-friendly demonstrations; a modified PointNet++ with multi-radius set abstraction over human and object point clouds plus flow then decodes coordinated base-arm actions.
Key results
- Ours reaches 63.8% / 53.4% / 77.8% success on m0 / n0 / s0 vs 46.8 / 32.9 / 61.1 for GenH2R (reprod.), a gain of at least 15 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MobileH2R instead of GenH2R (reprod.) | Mobile H2R handover · m0, n0, s0 | success rate: 46.80 -> 63.80; 32.90 -> 53.40; 61.11 -> 77.78 | Helped |
| Use MobileH2R instead of GenH2R (reprod.) | Real-world mobile handover · m0, n0 | success rate: 12 / 30 (40.0%) -> 24 / 30 (80.0%); 9 / 30 (30.0%) -> 19 / 30 (63.3%) | Helped |
| remove future obstacle avoidance (ablation) | MobileH2R demonstration generation · m0, n0, s0 | success rate: 63.8 -> 58.7; 53.4 -> 48.5; 77.8 -> 75.7 | Hurt |
| remove final pose constraints (ablation) | MobileH2R demonstration generation · m0, n0, s0 | success rate: 63.8 -> 58.4; 53.4 -> 43.2; 77.8 -> 69.4 | Hurt |
| remove imitation-friendly loss (ablation) | MobileH2R demonstration generation · m0, n0, s0 | success rate: 63.8 -> 56.8; 53.4 -> 46.4; 77.8 -> 56.9 | Hurt |
| replace vision neural loss with camera orientation-based loss (ablation) | MobileH2R demonstration generation · m0, n0, s0 | success rate: 63.8 -> 54.0; 53.4 -> 42.8; 77.8 -> 56.0 | Hurt |
| remove flow information (ablation) | MobileH2R policy · m0, n0, s0 | success rate: 63.8 -> 58.0; 53.4 -> 47.2; 77.8 -> 53.5 | Hurt |
| remove human information (ablation) | MobileH2R policy · m0, n0, s0 | success rate: 63.8 -> 51.3; 53.4 -> 48.7; 77.8 -> 57.6 | Hurt |
| decode base and arm actions separately instead of jointly (ablation) | MobileH2R policy · m0, n0, s0 | success rate: 63.8 -> 51.0; 53.4 -> 41.7; 77.8 -> 48.9 | Hurt |
| increase demonstrations from 10k to 100k (ablation) | MobileH2R trained on n0 · m0, n0, s0 | success rate: 63.8 -> 65.6; 53.4 -> 56.6; 77.8 -> 82.6 | Helped |
| reduce demonstrations from 10k to 1k (ablation) | MobileH2R trained on n0 · m0, n0, s0 | success rate: 63.8 -> 53.4; 53.4 -> 42.3; 77.8 -> 57.6 | Hurt |
| train on the less diverse m0 assets instead of n0 (ablation) | MobileH2R · m0, n0, s0 | success rate: 63.8 -> 62.0; 53.4 -> 45.3; 77.8 -> 76.4 | Mixed |
Surprising findings
- A camera-orientation heuristic loss that keeps the object centered in view reduces policy success (e.g., 42.8 vs 46.4 on n0), indicating hacky visibility designs can hurt imitation.
- Training on the simpler m0 dataset and testing on complex n0 drops success by 8.1%, while mocap-based s0 data gives a large drop of 34.6% on average, showing scale and diversity matter more than realism alone.
Limitations
- Real-world evaluation is limited to 6 objects in two settings with a user study rather than large-scale trials.
- Demonstration generation relies on privileged oracle scene knowledge (object pose, human poses, grasps) that is unavailable at deployment.
- Balancing demonstration safety and imitation-friendliness requires careful tuning; heuristic visibility losses can hurt performance.
Experimental setup
Datasets: MobileH2R-Sim (synthetic) ShapeNet Acronym DexGraspNet AMASS DexYCB
Benchmarks: MobileH2R-Sim m0 MobileH2R-Sim n0 MobileH2R-Sim s0 (DexYCB-based) real-world Galbot G1 sim-to-real user study
Metrics: success rate time average success (AS) human contact rate
Baselines: Grasp Selection + Trajectory Planning GenH2R GenH2R (reprod.)
Builds on: GenH2R CHOMP Guided Motion Diffusion PointNet++ HandoverSim
Closest papers by meaning
- GenH2R: Learning Generalizable Human-to-Robot Handover via Scalable Simulation Demonstration and Imitation (CVPR 2024 · similarity 0.95)
- MoManipVLA: Transferring Vision-language-action Models for General Mobile Manipulation (CVPR 2025 · similarity 0.86)
- Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation (ICLR 2025 · similarity 0.85)
- Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training (NeurIPS 2025 · similarity 0.85)
- STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learning (ICLR 2025 · similarity 0.85)
- Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Training (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.