Research map / Point clouds / Point cloud classification
GMAP: Generalized Manipulation of Articulated Objects in Robotic Using Pre-trained Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GMAP, a pipeline that segments articulated parts, estimates joint parameters and point-level affordances with a multi-scale ViT point cloud encoder pre-trained via Point-MGE, and plans manipulation trajectories from commands.
Problem
Robotic perception and manipulation of articulated objects is handled only piecemeal by existing methods, and annotated 3D data is scarce, limiting generalization of perception models.
Approach
Integrates part segmentation, joint parameter estimation, and affordance proposals into one command-to-trajectory pipeline, using a multi-scale ViT point cloud encoder (MSFE) pre-trained with masked point generative SSL on ShapeNet and then fine-tuned on articulated objects.
Key results
- Segmentation mIoU of 79.6% on PartNet-Mobility and 81.4% on Shape2Motion, ahead of Cart (72.3%) and MARS-style baselines.
- Revolute joint orientation error of 0.45° on PartNet-Mobility vs 1.58° for MARS.
- Instruction-based push manipulation success of 36.94% and pull success of 21.72% in Sapien, above AdaAfford (27.71% push).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add pre-training (ablation) | GMAP · PartNet-Mobility | segmentation mIoU (%): 77.0 -> 79.6 | Helped |
| Add pre-training (ablation) | GMAP · Shape2Motion | segmentation mIoU (%): 77.6 -> 81.4 | Helped |
| Add pre-training (ablation) | GMAP · PartNet-Mobility | revolute joint orientation error: 1.06° -> 0.45° | Helped |
| Add pre-training (ablation) | GMAP · Shape2Motion | revolute joint orientation error: 0.94° -> 0.39° | Helped |
| Add pre-training (ablation) | GMAP · PartNet-Mobility | prismatic joint orientation error: 1.65° -> 0.35° | Helped |
| Add pre-training (ablation) | GMAP · Shape2Motion | prismatic joint orientation error: 1.48° -> 0.49° | Helped |
| Add pre-training (ablation) | GMAP · PartNet-Mobility | revolute joint position error: 0.05 -> 0.04 | Hurt |
| Add dynamic trajectory planning from joint parameters instead of predefined actions (ablation) | GMAP Afford-Net · Sapien manipulation, five categories | pushing success (%): 18.23 -> 36.94 | Helped |
| Add dynamic trajectory planning from joint parameters instead of predefined actions (ablation) | GMAP Afford-Net · Sapien manipulation, five categories | pulling success (%): 9.39 -> 21.72 | Helped |
Surprising findings
- Position estimation for revolute joints on PartNet-Mobility slightly trails MARS (0.04 vs 0.03 in Table 1 is comparable but MARS is marginally better), so gains are not uniform across metrics.
- Our-Afford (affordance module alone) slightly lags AdaAfford in push success, indicating the trajectory planning component drives most of the gain.
Limitations
- Accuracy of interaction affordance proposals remains a challenge, as stated by the authors.
- Real-world experiments use only three objects and the method relies on compliance control to handle estimation errors.
- Generalization evaluated on only one training category (cabinet) and five unseen categories.
Experimental setup
Datasets: ShapeNet PartNet-Mobility Shape2Motion
Benchmarks: PartNet-Mobility Shape2Motion Sapien manipulation tests
Metrics: mIoU orientation error position error state error pushing success rate pulling success rate
Baselines: RPM-Net ANCSH Ditto Cart MARS Where2Act AdaAfford
Builds on: Point-MGE Where2Act Cart PointNet++ Ditto
Closest papers by meaning
- OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints (CVPR 2025 · similarity 0.87)
- AdaManip: Adaptive Articulated Object Manipulation Environments and Policy Learning (ICLR 2025 · similarity 0.87)
- IAAO: Interactive Affordance Learning for Articulated Objects in 3D Environments (CVPR 2025 · similarity 0.86)
- General Articulated Objects Manipulation in Real Images via Part-Aware Diffusion Process (NeurIPS 2024 · similarity 0.86)
- JODA: Composable Joint Dynamics for Articulated Objects (NeurIPS 2026 · similarity 0.86)
- AffordDP: Generalizable Diffusion Policy with Transferable Affordance (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.