Research map / Reinforcement learning / Visual and representation RL
DyMoDreamer: World Modeling with Dynamic Modulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DyMoDreamer, a DreamerV3-based world model that encodes inter-frame differencing masks as a dynamic modulator integrated into the RSSM, reporting SOTA on Atari 100k and DMC and a gain on Crafter.
Problem
Sample inefficiency in model-based RL: conventional RSSM and transformer world models process observations holistically and struggle to capture small, decision-relevant dynamic objects in visual tasks.
Approach
Computes backward inter-frame differencing masks (dilated to cover neighborhoods) to form differential observations, encodes them into stochastic categorical dynamic modulators that are fed into the RSSM sequence model and reward/continue heads, and adds an inter-frame KL differential regularizer.
Key results
- Atari 100k mean human-normalized score of 156.6% vs DIAMOND 146% and DreamerV3 125%.
- DeepMind Visual Control Suite task mean 832 vs TWISTER 801.8 and DreamerV3 786 over 20 tasks at 1M steps.
- Crafter return at 1M steps of 10.3 vs DreamerV3 9.4 (about 9.5% gain).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dynamic modulation to DreamerV3 | DreamerV3 · Atari 100k | HNS Mean: 125% -> 156.6% | Helped |
| add dynamic modulation to DreamerV3 | DreamerV3 · DeepMind Visual Control Suite | Task Mean: 786 -> 832 | Helped |
| add dynamic modulation to DreamerV3 | DreamerV3 · Crafter @1M | Return: 9.4 -> 10.3 | Helped |
| add dynamic modulation to DreamerV3 | DreamerV3 · DeepMind Visual Control Suite | Task Median: 861 -> 871 | Helped |
| integrate dynamic modulation into the RSSM sequence model (ablation) | DyMoDreamer · Boxing, Krull, Pong, and Road Runner | performance | Helped |
| add differential divergence regularization (ablation) | DyMoDreamer · Boxing, Krull, Pong, and Road Runner | performance | Helped |
| increase the dimensionality of the stochastic representation alone (ablation) | DyMoDreamer · Atari games | performance | No clear effect |
| replace frame differencing with moving average differencing (ablation) | DyMoDreamer · Battle Zone | score: 16240 -> 20351 | Helped |
| replace frame differencing with multi-frame logical differencing (ablation) | DyMoDreamer · Road Runner | score: 20972 -> 21378 | Helped |
Surprising findings
- Reconstruction loss on imagined trajectories shows no significant reduction, yet hallucinations on dynamic patterns are reduced versus DreamerV3.
- DyMoDreamer scores lower than DreamerV3 on some games such as Qbert (1736 vs 2921) and Chopper Command (740 vs 2222).
Limitations
- Recurrent latent models offer limited long-term memory and compounding errors in long-horizon imagination.
- Dynamic-modulation gains are not uniform; several games underperform DreamerV3 or other baselines.
- Generalization to real-world domains and robustness under distribution shift remain untested.
Experimental setup
Datasets: Atari 100k DeepMind Visual Control Suite Crafter DeepMind Proprio
Benchmarks: Atari 100k DeepMind Visual Control Suite Crafter DeepMind Proprio
Metrics: human-normalized score mean human-normalized score median task mean task median return @1M
Baselines: DreamerV3 OC-STORM DIAMOND HarmonyDream TD-MPC2 TWISTER IRIS Delta-IRIS STORM
Builds on: DreamerV3 RSSM latent flow straight-through gradients
Closest papers by meaning
- Seek Commonality but Preserve Differences: Dissected Dynamics Modeling for Multi-modal Visual RL (NeurIPS 2024 · similarity 0.89)
- ReCoRe: Regularized Contrastive Representation Learning of World Model (CVPR 2024 · similarity 0.87)
- DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization (ICLR 2024 · similarity 0.87)
- Neural Motion Simulator Pushing the Limit of World Models in Reinforcement Learning (CVPR 2025 · similarity 0.87)
- HarmonyDream: Task Harmonization Inside World Models (ICML 2024 · similarity 0.87)
- VIGOR: Zero-Shot Visual Generalization via Latent-Space Consistency in Model-Based Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.