Research map / Reinforcement learning / Visual and representation RL
Learning Latent Dynamic Robust Representations for World Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HRSSM, a hybrid RSSM on DreamerV3 that combines cube masking, latent (not pixel) reconstruction, and a bisimulation-style similarity loss to learn exogenous-noise-robust world-model representations.
Problem
Dreamer-style visual model-based RL struggles when pixel observations contain exogenous or irrelevant distractor noise, since pixel reconstruction emphasizes irrelevant details and approximate dynamics undermine bisimulation-based representations.
Approach
A hybrid RSSM with a masked-observation branch and an EMA raw branch sharing history; the masked branch is trained to match raw-branch latents (latent reconstruction, no pixel decoder) together with a bisimulation-style reward/dynamics similarity loss, and gradients flow only through the mask branch.
Key results
- On Realistic ManiSkill Lift Cube at 100k steps, the method scores 284±85 versus DreamerV3 167±45 and TIA 274±173.
- On Turn Faucet at 100k steps, the method scores 278±97 versus DreamerV3 138±83 and RePo 92±88.
- Model has 17.54M parameters versus 18.22M for DreamerV3, and wall-clock training time is reported as comparable to DreamerV3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add HRSSM with masking-based latent reconstruction and the similarity objective | DreamerV3 · Realistic ManiSkill2: Lift Cube, 100K training steps | average score: 284$\pm$85 vs 274$\pm$173 | Helped |
| Add HRSSM with masking-based latent reconstruction and the similarity objective | DreamerV3 · Realistic ManiSkill2: Turn Faucet, 100K training steps | average score: 138$\pm$83 -> 278$\pm$97 | Helped |
| Add masking-based latent reconstruction and the mask branch (ablation) | DreamerV3 with similarity objective · DMC tasks with exogenous noise | mean return | Helped |
| Add the bisimulation-guided similarity loss (ablation) | HRSSM · DMC tasks with exogenous noise | mean return | Helped |
| Add only the similarity-based objective to DreamerV3 (ablation) | DreamerV3 · DMC tasks with exogenous noise | sample efficiency | Mixed |
| Use HRSSM with masking and similarity objectives | DreamerV3 · DMC default setting | mean return | Helped |
| Use HRSSM with masking and similarity objectives | DreamerV3 · DMC distraction setting | mean return | Mixed |
Surprising findings
- Adding only the similarity (bisimulation) loss to DreamerV3 does not consistently improve sample efficiency and often gives the lowest performance.
- The method underperforms DeepMDP and RePo on cartpole_swingup in the distraction setting, attributed to random masking hiding the small pole.
Limitations
- Random masking is not task-specific and may occlude task-critical elements (e.g., small pole in cartpole_swingup), slightly hurting performance.
- Theoretical counterexample suggests bisimulation alone is insufficient for compact representations under exogenous noise; relies on approximate dynamics support assumptions.
Experimental setup
Datasets: DeepMind Control Suite Distracted DMControl (Kinetics-400 grayscale videos) ManiSkill2 Matterport3D scenes
Benchmarks: DeepMind Control Suite (default) Distracted DMControl Realistic ManiSkill2 (Lift Cube, Turn Faucet with Matterport backgrounds)
Metrics: average return mean return at 100k steps IQM with bootstrapped confidence intervals standard deviation
Baselines: DreamerV3 TIA Denoised MDP RePo DBC SimSR TD-MPC
Builds on: DreamerV3 RSSM bisimulation metrics DBC BYOL-style EMA encoder
Closest papers by meaning
- Learning Fused State Representations for Control from Multi-View Observations (ICML 2025 · similarity 0.89)
- Zero-shot World Models via Search in Memory (NeurIPS 2025 · similarity 0.89)
- Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization (NeurIPS 2025 · similarity 0.88)
- Learning Visual Feature-Based World Models via Residual Latent Action (NeurIPS 2026 · similarity 0.87)
- The Surprising Ineffectiveness of Pre-Trained Visual Representations for Model-Based Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
- Policy-shaped prediction: avoiding distractions in model-based reinforcement learning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.