Research map / Reinforcement learning / Visual and representation RL
DMWM: Dual-Mind World Model with Long-Term Imagination
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DMWM, a dual-process world model that pairs a DreamerV3-style RSSM (System 1) with a logic-integrated neural network (System 2) that imposes logical regularization over imagined state-action sequences, evaluated with actor-critic and gradient MPC on DMControl and robotic tasks.
Problem
RSSM-based world models accumulate prediction errors over long imagination horizons and lack logical consistency, limiting long-term planning and sample efficiency.
Approach
Combine an RSSM System 1 with a LINN-based System 2 that performs hierarchical deep logical reasoning (conjunction, implication, recursive depth-alpha implications) over state and action logic embeddings, with bidirectional feedback: S2 constrains S1 imagination via a logical ELBO term, and S1 transitions update S2's logic.
Key results
- DMWM reports 14.3% logic-consistency improvement over Dreamer on DMC at horizon H=30 (e.g., Cartpole Balance 0.727 vs 0.683 for Dreamer).
- Claims an average 5.5-fold improvement in test return under limited environment trials versus baselines.
- Reports an average 32% test-return improvement under limited data versus Dreamer and GD-MPC, and about 120% for H>30 on complex control tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LINN-S2 logical reasoning to the RSSM world model | DMWM · DMC tasks | logical consistency: 14.3\% | Helped |
| Add LINN-S2 logical reasoning to the RSSM world model | DMWM · DMC tasks under limited environment trials | test return: 5.5-fold | Helped |
| Add LINN-S2 logical reasoning to the RSSM world model | DMWM · Complex DMC tasks under limited environment steps | test return: 32\% | Helped |
| Add LINN-S2 logical reasoning to the RSSM world model | DMWM · Complex control tasks, H>30 | test return: 120\% | Helped |
| Increase logical inference depth (ablation) | DMWM · DMC long-horizon decision making | test return | Helped |
Limitations
- Relies on predefined simple domain-specific logical rules that may be ambiguous or evolving.
- Larger logic inference depth gives diminishing returns with increased computational overhead.
Experimental setup
Datasets: DMControl (20 tasks) ManiSkill2 (4 tasks) MyoSuite (4 tasks)
Benchmarks: DMControl ManiSkill2 MyoSuite
Metrics: logic consistency (mean and variance) test return return under limited environment trials return under limited environment steps
Baselines: DreamerV3 Dreamer-enabled Grad-MPC TD-MPC2 Hieros HRSSM
Builds on: DreamerV3 LINN TD-MPC2 Grad-MPC
Closest papers by meaning
- RIG: Synergizing Reasoning and Imagination in End-to-End Generalist Policy (ICLR 2026 · similarity 0.87)
- Agent Planning with World Knowledge Model (NeurIPS 2024 · similarity 0.86)
- Open-World Reinforcement Learning over Long Short-Term Imagination (ICLR 2025 · similarity 0.85)
- Evolving Minds: Logic-Informed Inference from Temporal Action Patterns (ICML 2025 · similarity 0.85)
- HaM-World: Soft-Hamiltonian World Models with Selective Memory for Planning (NeurIPS 2026 · similarity 0.85)
- Mastering Memory Tasks with World Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.