Research map / Reinforcement learning / Visual and representation RL
Modeling Unseen Environments with Language-guided Composable Causal Components in Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes WM3C, a language-guided world model that identifies composable causal latent components (block-wise identifiable) and reuses them for RL generalization to unseen task compositions.
Problem
Generalizing RL policies to novel, unseen environments with changed dynamics, where agents overfit to training task structure and existing invariant-representation, meta-RL, and world-model methods do not exploit compositional causal structure.
Approach
Use language components as an additional compositional modality to decompose the latent state into language-controlled causal components, with a block-wise identifiability theorem, and learn a world model (built on DreamerV3) with component-wise KL, mutual-information constraints, and adaptive sparse masks so that only dynamics modules need adapting to new recombinations.
Key results
- On synthetic data, WM3C achieves diagonal R^2 above 0.9 for identified components and outperforms iVAE, TCL, TDRL, NCTRL, DreamerV3 and TD-MPC2 in identification.
- On Meta-World, WM3C (MAE) shows better training data efficiency than DreamerV3 and MT-SAC across 18 training tasks.
- On 7 recombination test tasks, WM3C variants adapting only the dynamics module achieve higher success than fully fine-tuned DreamerV3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use language-guided composable causal components | WM3C · Synthetic data, known tasks | R² | Helped |
| use language-guided composable causal components | WM3C · Synthetic data, unseen task combinations | R² | Helped |
| use language-guided composable world modeling | WM3C · Meta-World training tasks | success rate | Helped |
| use MAE instead of CNN | WM3C · Meta-World training tasks | success rate | Helped |
| adapt only the dynamics module rather than fine-tuning all parameters | WM3C · Meta-World, 7 unseen tasks recombining known components | success rate | Helped |
| adapt only the dynamics module for tasks containing unknown components | WM3C · Meta-World, coffee-button and handle-press-side | success rate | Mixed |
Surprising findings
- Adapting only the dynamics module of WM3C was worse than DreamerV3 on both tasks with a novel component (coffee-button, handle-press-side), with WM3C better in only one case.
Limitations
- Identifiability conditions are not fully met in practice; the paper notes they are not always easily satisfied in applications.
- Adaptation via dynamics-module tuning does not always suffice for tasks with unknown components.
- Scalability to large numbers of language components is left to future work.
Experimental setup
Datasets: Meta-World synthetic numerical simulation dataset
Benchmarks: Meta-World synthetic numerical simulation
Metrics: R^2 (kernel ridge regression) success rate
Baselines: DreamerV3 TD-MPC2 MT-SAC iVAE TCL TDRL NCTRL
Builds on: DreamerV3 iVAE TDRL NCTRL MAE
Closest papers by meaning
- Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization (NeurIPS 2025 · similarity 0.88)
- Towards Generalizable Reinforcement Learning via Causality-Guided Self-Adaptive Representations (ICLR 2025 · similarity 0.88)
- Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following (NeurIPS 2025 · similarity 0.88)
- RoboDreamer: Learning Compositional World Models for Robot Imagination (ICML 2024 · similarity 0.88)
- Curious Causality-Seeking Agents Learn Meta Causal World (NeurIPS 2025 · similarity 0.87)
- CraftFactory: A Conditioned Control Policy Benchmark for Compositional Generalization (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.