Research map / Reinforcement learning / Visual and representation RL
Learning Interactive World Model for Object-Centric Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FIOC-WM, an object-centric world model that learns object interactions and attribute (static/dynamic) factorization from pixel embeddings, and trains a hierarchical policy that composes learned interaction primitives for long-horizon control.
Problem
Object-centric RL methods factor state by objects but leave interactions implicit, limiting sample efficiency and generalization of policy learning in long-horizon robotic and embodied tasks.
Approach
Learn a two-level factorization (object slots and per-object static/dynamic attributes) plus latent time-varying interaction graphs within a world model built on pre-trained visual embeddings, then use a high-level policy to select interaction subgoal graphs and a low-level interaction policy (MPC or PPO) to execute them.
Key results
- On Libero compositional generalization, FIOC reaches 0.70 success rate vs 0.65 for EIT and 0.58 for Dreamer-V3.
- Across Gym Fetch, Franka Kitchen, i-Gibson and Libero single-task results, FIOC is best in 5 of 8 tasks per the appendix claim.
- LPIPS on Fetch improves from 0.009 (DINO-WM) to 0.007 (FIOC).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FIOC instead of the strongest listed baseline | world-model reconstruction · Fetch | LPIPS: 0.009 -> 0.007 | Helped |
| Use FIOC instead of the strongest listed baseline | world-model reconstruction · Kitchen | LPIPS: 0.035 -> 0.038 | Hurt |
| Use FIOC instead of EIT | policy learning · Attribute generalization, i-Gibson | success rate: 0.70 -> 0.79 | Helped |
| Use FIOC instead of the strongest listed baseline | policy learning · Attribute generalization, Push & Switch | success rate: 0.95 -> 0.91 | Hurt |
| Add static/dynamic state factorization (ablation) | FIOC · Libero, compositional generalization | success rate: 0.64 -> 0.70 | Helped |
| Add learned interaction modeling rather than a fully connected graph (ablation) | FIOC · Libero, compositional generalization | success rate: 0.52 -> 0.70 | Helped |
| Use pre-trained policies instead of random actions in offline learning (ablation) | FIOC · Libero, compositional generalization | success rate: 0.48 -> 0.70 | Helped |
| Add the high-level hierarchical policy (ablation) | FIOC · Libero, compositional generalization | success rate: 0.42 -> 0.70 | Helped |
| Pre-train the low-level policy offline (ablation) | FIOC · Libero, compositional generalization | success rate: 0.59 -> 0.70 | Helped |
| Add the diversity term to high-level policy learning (ablation) | FIOC · Libero, compositional generalization | success rate: 0.50 -> 0.70 | Helped |
| Tune the world model online (ablation) | FIOC · Gym Fetch Task 1 | success rate: 0.95 \pm 0.03 -> 0.96 \pm 0.02 | No clear effect |
| Tune the world model online (ablation) | FIOC · Franka Kitchen Task 1 | success rate: 0.82 \pm 0.04 -> 0.79 \pm 0.06 | Mixed |
Surprising findings
- Removing the interaction modeling drops Libero single-task success from 0.81 to 0.63, the largest drop among world-model components.
- Online world-model fine-tuning does not consistently help: several tasks decline (e.g., Franka Kitchen Task 1 from 0.82 to 0.79).
Limitations
- Relies on a pretrained object-centric model for object discovery.
- Interaction models primarily generalize to seen object categories.
- Validated only on simulated benchmarks, not real robots.
Experimental setup
Datasets: SpritesWorld OpenAI-Gym Fetch Franka Kitchen i-Gibson LIBERO
Benchmarks: SpritesWorld OpenAI-Gym Fetch Franka Kitchen i-Gibson LIBERO
Metrics: LPIPS normalized SHD MSE (linear probing) success rate
Baselines: DreamerV3 TD-MPC2 EIT DINO-WM
Builds on: DINO-WM SKILD slot attention neural relational inference PPO
Closest papers by meaning
- Efficient Exploration and Discriminative World Model Learning with an Object-Centric Abstraction (ICLR 2025 · similarity 0.88)
- SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels (ICML 2025 · similarity 0.87)
- Learning Interactive Real-World Simulators (ICLR 2024 · similarity 0.86)
- Modeling Unseen Environments with Language-guided Composable Causal Components in Reinforcement Learning (ICLR 2025 · similarity 0.86)
- DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning (ICML 2025 · similarity 0.86)
- Entity-Centric Reinforcement Learning for Object Manipulation from Pixels (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.