Research map / Detection and segmentation / Open-set and few-shot detection
Unsupervised Object Interaction Learning with Counterfactual Dynamics Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes COIL, an unsupervised RL method that learns goal-conditioned object-interaction skills via a counterfactual intrinsic reward computed from a forward model or successor features on entity-centric states.
Problem
Learning primitive object-object interaction skills in reward-free, entity-centric continuous control environments, where naive exploration tends to focus on the agent or single objects.
Approach
Define an interaction reward as the change in object A's predicted future state when object B is counterfactually intervened on (replaced with a random object state from the replay buffer), using either a one-step forward model (COIL-Forward) or entity-centric successor features (COIL-SF); the goal is a pair of object indices.
Key results
- On Stacking Box, COIL agents converge to a success rate of approximately 1.0 while curiosity-based baselines plateau at lower upper bounds.
- On Magnetic Blocks, COIL-SF is the only algorithm that successfully learns object interaction skills; Forward-Curiosity, SID, and RND grab objects but rarely connect them.
- COIL-SF fine-tuned from a 4-object pretrained policy generalizes to unseen object sizes and to 6-object settings, outperforming training from scratch.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use successor features instead of a one-step forward model for the counterfactual interaction reward (ablation) | COIL · Magnetic Blocks | success rate | Helped |
| Use a one-step forward model instead of successor features for the counterfactual interaction reward (ablation) | COIL · Magnetic Blocks | success rate | Hurt |
| Use a one-step forward model instead of successor features for the counterfactual interaction reward (ablation) | COIL · Stacking Box, 4 objects | success rate | Helped |
| Fine-tune a COIL-SF agent pretrained with 4 objects at 100% object size | COIL-SF · Magnetic Blocks, 4 objects, 33% object size | success rate | Helped |
| Fine-tune a COIL-SF agent pretrained with 4 objects at 100% object size | COIL-SF · Magnetic Blocks, 6 objects | success rate | Helped |
Surprising findings
- Sparse-GT (using ground-truth interaction reward) fails to learn even the basic task of grabbing an object in Magnetic Blocks.
- COIL-Forward outperforms COIL-SF in Stacking Box with 4 objects, but COIL-Forward is not effective in Magnetic Blocks where COIL-SF is needed.
Limitations
- Relies on a factorized (entity-centric) state representation, not directly applicable to pixel observations without object representation learning.
- Does not consider diverse modes of interaction skills.
- Goals are categorical object indices rather than continuous object representations.
Experimental setup
Datasets: Stacking Box (simulated) Magnetic Blocks (simulated)
Benchmarks: Stacking Box Magnetic Blocks
Metrics: success rate average reward per state label
Baselines: Sparse-GT Forward-Curiosity SID RND
Builds on: SAC successor features SID RND Relational forward model (C-SWM)
Closest papers by meaning
- SkiLD: Unsupervised Skill Discovery Guided by Factor Interactions (NeurIPS 2024 · similarity 0.86)
- SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels (ICML 2025 · similarity 0.85)
- Entity-Centric Reinforcement Learning for Object Manipulation from Pixels (ICLR 2024 · similarity 0.85)
- Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning (NeurIPS 2024 · similarity 0.85)
- Learning Interactive World Model for Object-Centric Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
- $\textit{HiMaCon:}$ Discovering Hierarchical Manipulation Concepts from Unlabeled Multi-Modal Data (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.