Research map / Reinforcement learning / Visual and representation RL
Geometry-aware RL for Manipulation of Varying Shapes and Deformable Objects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HEPi, an SE(3)-equivariant heterogeneous graph policy (built on PONITA-style EMPN) trained with TRPL on a new suite of seven rigid and deformable manipulation tasks in IsaacLab, outperforming Transformer and homogeneous EMPN baselines.
Problem
Reinforcement learning for manipulating rigid objects with varying geometries and deformable objects (rope, cloth) where initial and target configurations are sampled uniformly in 3D, creating large search spaces that generalize poorly across poses.
Approach
Model manipulation as a heterogeneous graph with separate actuator and object sub-graphs, use SE(3)-equivariant message passing with type-specific kernels for each edge type, and aggregate object information to actuators via a virtual-node-style global update, optimized with trust-region projection layers instead of PPO.
Key results
- HEPi matches Transformer on the simpler 2D rigid tasks but outperforms it on rigid-insertion-two-agents-3D and cloth-hanging-3D, with a significant advantage over EMPN and Transformer on cloth-hanging-3D.
- Both EMPN and HEPi show better sample efficiency than the Transformer across tasks due to exploiting symmetry.
- Attention as aggregation gives no noticeable benefit and nearly doubles HEPi training time.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HEPi instead of Transformer | Manipulation tasks · Rigid-Insertion-Two-Agents-3D | average return | Helped |
| Use HEPi instead of EMPN | Manipulation tasks · Rigid-Pushing-2D | average return | Helped |
| Use HEPi instead of EMPN | Manipulation tasks · Rigid-Insertion-Two-Agents-3D | average return | Helped |
| Use HEPi instead of Transformer | Manipulation tasks · Rope-Closing and Rope-Shaping | generalization | Helped |
| Use HEPi instead of EMPN | Manipulation tasks · Rope-Closing and Rope-Shaping | average return | No clear effect |
| Use HEPi instead of Transformer and EMPN | Manipulation tasks · Cloth-Hanging-3D | average return | Helped |
| Use heterogeneous models instead of homogeneous models | Cloth-Hanging policies · Cloth-Hanging | final return | Helped |
| Use EMPN with HEPi instead of HeteroGNN | Cloth-Hanging policies · Cloth-Hanging | sample efficiency | Helped |
| Train on a smaller sample space | Cloth-Hanging policies · Cloth-Hanging | final return | Helped |
| Add attention as the aggregation function (ablation) | Homogeneous and heterogeneous graph networks · Manipulation tasks | performance | No clear effect |
| Add attention as the aggregation function (ablation) | HEPi · Manipulation tasks | training time: almost doubled | Hurt |
| Use TRPL instead of PPO (ablation) | HEPi and Transformer policies · Cloth-Hanging-3D | training stability | Helped |
Surprising findings
- Attention aggregation does not improve performance and roughly doubles HEPi training time.
- A well-tuned PPO performs comparably to TRPL on 2D tasks, with TRPL advantages mainly in high-dimensional 3D tasks.
Limitations
- Robot body abstracted away; only end-effector control is modeled.
- Assumes keypoint coordinates are available as observations rather than extracted from vision.
- Equivariance over orientation sampling on S^2 is only approximate.
Experimental setup
Datasets: LASA dataset
Benchmarks: Rigid-Sliding Rigid-Pushing Rigid-Insertion Rigid-Insertion-Two-Agents Rope-Closing Rope-Shaping Cloth-Hanging
Metrics: undiscounted return interquartile mean with 95% CI sample efficiency generalization to unseen objects
Baselines: Transformer policy EMPN HeteroGNN naive GNN PPO GAT
Builds on: PONITA Trust Region Projection Layers PPO MPNN + virtual node DeepSets
Closest papers by meaning
- DeformGen: Dynamics-Based Topology Augmentation for Deformable Manipulation Policy Learning (NeurIPS 2026 · similarity 0.87)
- Subequivariant Graph Reinforcement Learning in 3D Environments (ICML 2023 · similarity 0.87)
- FlatLab: A Unified Methodology Framework and Simulation-Based Benchmark for Robotic Manipulation of Flat Objects (ICML 2026 · similarity 0.86)
- Deep SE(3)-Equivariant Geometric Reasoning for Precise Placement Tasks (ICLR 2024 · similarity 0.85)
- Entity-Centric Reinforcement Learning for Object Manipulation from Pixels (ICLR 2024 · similarity 0.84)
- Spatial-Temporal Graph Diffusion Policy with Kinematic Modeling for Bimanual Robotic Manipulation (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.