Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Rapid Motor Adaptation for Robotic Manipulator Arms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends Rapid Motor Adaptation to general robot-arm manipulation by adding category/instance object embeddings to privileged info and a depth-CNN adapter, evaluated in four ManiSkill2 tasks.
Problem
Generalizable manipulation across object shape, density, friction and external disturbances; RMA was proposed for locomotion and in-hand rotation and proprioception alone cannot infer object identity before grasping.
Approach
Condition the policy on privileged environment embeddings that include learnable object instance and category dictionaries, then train a depth-image plus proprioceptive-history adapter via L2 regression to estimate those embeddings for real-time adaptive manipulation with a Franka arm.
Key results
- On Pick & Place (YCB), RMA2 reaches 73.8% success vs 75.4% for the privileged Oracle and 35.2% for DR+Vi; DR and ADR near 0-2%.
- On Faucet Turning, RMA2 reaches 62.7% success vs 76.2% Oracle and 51.8% ADR.
- On Pick & Place with unseen EGAD objects, RMA2 reaches 90.5% success, higher than its YCB result, while DR+Vi stays at 35.5%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RMA2 instead of DR+Vi | Pick & Place (YCB) · Pick & Place (YCB) | success rate (SR): 35.2 ± 3.7 -> 73.8 ± 4.5 | Helped |
| use RMA2 instead of ADR | Faucet Turning · Faucet Turning | success rate (SR): 51.8 ± 1.8 -> 62.7 ± 0.5 | Helped |
| use RMA2 instead of DR | Peg Insertion · Peg Insertion | success rate (SR): 45.6 ± 21.6 -> 51.6 ± 6.7 | Helped |
| use RMA2 instead of DR+Vi | Pick & Place (EGAD) · Pick & Place (EGAD) | success rate (SR): 35.5 ± 4.5 -> 90.5 ± 2.8 | Helped |
| add object embeddings (ablation) | RMA2 · Pick & Place (YCB) | success rate (SR): 70.4 ± 0.4 -> 73.8 ± 4.5 | Helped |
| add depth vision to adaptation (ablation) | RMA2 · Pick & Place (YCB) | success rate (SR): 68.1 ± 0.8 -> 73.8 ± 4.5 | Helped |
| add object embeddings (ablation) | RMA2 · Faucet Turning | success rate (SR): 57.6 ± 0.4 -> 62.7 ± 0.5 | Helped |
| add depth vision to adaptation (ablation) | RMA2 · Faucet Turning | success rate (SR): 47.0 ± 0.7 -> 62.7 ± 0.5 | Helped |
| add depth vision to adaptation (ablation) | RMA2 · Peg Insertion | success rate (SR): 48.4 ± 9.8 -> 51.6 ± 6.7 | Helped |
| add object embeddings (ablation) | RMA2 · Pick & Place (EGAD) | success rate (SR): 88.1 ± 0.1 -> 90.5 ± 2.8 | Helped |
| add depth vision to adaptation (ablation) | RMA2 · Pick & Place (EGAD) | success rate (SR): 87.5 ± 3.4 -> 90.5 ± 2.8 | Helped |
Surprising findings
- RMA2 transfers from YCB to EGAD objects with higher success (90.5%) than on the training YCB objects (73.8%).
- Vision-based DR+Vi fails completely on Peg Insertion (0.0%) due to timeouts before any progress.
Limitations
- Gap to Oracle remains on Faucet Turning (62.7% vs 76.2%), attributed to moving gripper camera.
- Policies trained separately per task; no multi-task agent explored.
- Only simulation results; no real-world deployment.
Experimental setup
Datasets: YCB EGAD PartNet-Mobility
Benchmarks: ManiSkill2 Pick & Place (YCB) ManiSkill2 Pick & Place (EGAD) ManiSkill2 Faucet Turning ManiSkill2 Peg Insertion
Metrics: success rate episode length
Baselines: Domain Randomization (DR) DR+Vi Automatic Domain Randomization (ADR) NoOE NoVA Oracle Adaptation
Builds on: Rapid Motor Adaptation Automatic Domain Randomization Proximal Policy Optimization ManiSkill2
Closest papers by meaning
- SRSA: Skill Retrieval and Adaptation for Robotic Assembly Tasks (ICLR 2025 · similarity 0.85)
- PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.85)
- ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR 2025 · similarity 0.85)
- Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation (ICLR 2025 · similarity 0.85)
- Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training (NeurIPS 2025 · similarity 0.85)
- Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasets (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.