Research map / Reinforcement learning / Visual and representation RL
Entity-Centric Reinforcement Learning for Object Manipulation from Pixels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Entity Interaction Transformer (EIT) policies and Q-functions over DLP object-centric particles from multiple views, trained with TD3+HER for goal-conditioned multi-cube manipulation from pixels, with a theoretical compositional generalization result.
Problem
Goal-conditioned multi-object manipulation from raw images is hard for RL due to the curse of dimensionality and the lack of structured, aligned entity representations; generalization across object counts is also unclear.
Approach
Combine a lean pretrained unsupervised object-centric image representation (DLP particles from multiple viewpoints) with a Transformer-based policy/Q-function (EIT) that models state-goal and entity-entity interactions without requiring matched entities, plus a Chamfer-distance image reward and a self-attention compositional generalization result.
Key results
- On 3-Cubes with GT reward, Ours (State) reaches 0.991 success on 2-Cubes and 0.978 on 3-Cubes; Ours (Image) reaches 0.968 and 0.919 respectively.
- On Adjacent-Goals, Ours (Image) achieves success rate 0.710 versus 0.716 for SMORL (State) and Ours (State) 0.963.
- An agent trained on 3 cubes generalizes zero-shot to 6 cubes with success rate 0.398 and decay in average return roughly linear in cube count.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EIT to model interactions between entities instead of SMORL's sequential, independent object approach | ECRL · Adjacent-Goals, Small-Table, and Ordered-Push | Success rate | Helped |
| Use EIT with image inputs instead of SMORL with state inputs | ECRL · Ordered-Push, Small-Table, and Adjacent-Goals | Success rate | Helped |
| Train on 3 cubes rather than 1 or 2 cubes | ECRL · N-Cubes compositional generalization | Maximum object distance: 2 & 0.024 & 0.256; 3 & 0.034 & 0.287 | Helped |
| Use multi-view rather than single-view image inputs (ablation) | ECRL · Multi-object manipulation | Sample efficiency | Helped |
| Add the action as a separate entity to the Q-function Transformer rather than concatenating it after the final Transformer block (ablation) | ECRL · State and image observations | Sample efficiency | Helped |
| Remove the separate action entity and concatenate the action after the final Transformer block (ablation) | ECRL · Image observations | Learning within the timestep budget | Hurt |
| Discard depth z_d and transparency z_t attributes (ablation) | ECRL · 2-Cubes | Success | No clear effect |
| Remove attention masking based on transparency (ablation) | ECRL · 2-Cubes | Success | No clear effect |
| Remove scale attribute z_s (ablation) | ECRL · 2-Cubes | Sample efficiency | Hurt |
Surprising findings
- An agent trained on 1 or 2 cubes fails to generalize beyond one additional object, whereas training on 3 cubes generalizes to 6 cubes reasonably well.
- Colors not seen in DLP pretraining are mapped to the closest known color, yet the agent still achieves non-negligible success.
Limitations
- Requires a pretrained DLP; pretraining on random-policy data may not suffice for more complex tasks.
- Chamfer (image-based) reward performs worse than ground-truth reward, suggesting real-robot reward design challenges.
- Zero-shot generalization to novel object shapes and properties is only partial.
Experimental setup
Datasets: random-policy rollouts in IsaacGym cube environments
Benchmarks: N-Cubes (IsaacGym) Adjacent-Goals Small-Table Ordered-Push Push-2T
Metrics: success rate success fraction max object distance average object distance average return
Baselines: Unstructured (beta-VAE + MLP) SMORL SMORL (State) Unstructured (State)
Builds on: Deep Latent Particles TD3 Hindsight Experience Replay SMORL
Closest papers by meaning
- SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels (ICML 2025 · similarity 0.88)
- Data Scaling Laws in Imitation Learning for Robotic Manipulation (ICLR 2025 · similarity 0.88)
- EC-Diffuser: Multi-Object Manipulation via Entity-Centric Behavior Generation (ICLR 2025 · similarity 0.87)
- Zero-Shot Robotic Manipulation with Pre-Trained Image-Editing Diffusion Models (ICLR 2024 · similarity 0.87)
- Multi-Stage Manipulation with Demonstration-Augmented Reward, Policy, and World Model Learning (ICML 2025 · similarity 0.87)
- OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.