Research map / Probabilistic and scientific modeling / Dynamical systems
Dynamics-Aligned Latent Imagination in Contextual World Models for Zero-Shot Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DALI, which adds a self-supervised forward-dynamics context encoder to DreamerV3 to infer latent contexts (gravity, string length, actuator strength) and reports zero-shot gains in DMC Ball-in-Cup and Walker extrapolation.
Problem
Zero-shot generalization in contextual MDPs when context variables (e.g., gravity, friction) are latent and not observed, so agents cannot condition on them directly.
Approach
A context encoder over short K-length observation-action windows is trained with a forward-dynamics loss (plus an optional cross-modal alignment loss to the RSSM posterior) and injected into DreamerV3 either shallowly (encoder input) or deeply (recurrent, reward, continue, decoder, actor-critic), with theoretical claims about mutual information and sample complexity.
Key results
- In Ball-in-Cup Featurized Extrapolation, DALI-S-chi reaches IQM 0.3720, a +87.9% gain over Dreamer-DR and above cRSSM-S and cRSSM-D.
- In Ball-in-Cup Pixel Extrapolation, DALI-S-chi reaches IQM 0.2730, a +96.4% gain over Dreamer-DR.
- In Walker Featurized Extrapolation, DALI-S reaches IQM 0.7810, a +4.0% gain over Dreamer-DR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| infer latent context with DALI-S-χ instead of Dreamer-DR | DreamerV3 on Ball-in-Cup · Featurized Extrapolation | IQM: +87.9% | Helped |
| infer latent context with DALI-S-χ instead of Dreamer-DR | DreamerV3 on Ball-in-Cup · Pixel Extrapolation | IQM: +96.4% | Helped |
| infer context with DALI-S-χ instead of ground-truth context conditioning | DreamerV3 on Ball-in-Cup · Featurized Extrapolation | IQM: 0.3720 vs. 0.2270, 0.2780 | Helped |
| infer context with DALI-S-χ instead of ground-truth context conditioning | DreamerV3 on Ball-in-Cup · Pixel Extrapolation | IQM: 0.2730 vs. 0.1870, 0.2420 | Helped |
| add cross-modal regularization to the forward-dynamics objective (ablation) | DALI-S on Ball-in-Cup · All regimes and modalities | IQM | Helped |
| add cross-modal regularization to the forward-dynamics objective (ablation) | DALI-S on Walker Walk · Most regimes and modalities | IQM: 0.7810 vs. 0.7770; 0.7580 vs. 0.7330 | Hurt |
| use inferred context instead of ground-truth context conditioning | DreamerV3 on Walker Walk · Featurized Extrapolation | IQM: 0.7810, 0.7770 vs. 0.7020, 0.7490 | Helped |
| use inferred context instead of ground-truth context conditioning | DreamerV3 on Ball-in-Cup · Pixel Mixed | IQM: 0.6250 vs. 0.6030 | Hurt |
| use inferred context instead of ground-truth context conditioning | DreamerV3 on Walker Walk · Pixel Extrapolation | IQM: 0.7770 vs. 0.7580 | Hurt |
| use DALI-S instead of Dreamer-DR | DreamerV3 on Walker Walk · Featurized Extrapolation | IQM: +4.0% | Helped |
Surprising findings
- Ground-truth context-aware baselines (cRSSM-S/D) are outperformed by the inferred-context DALI in extrapolation, which the authors attribute to overfitting to training contexts.
- Cross-modal regularization helps Ball-in-Cup but hurts Walker in most regimes, so the best loss variant is task-dependent.
Limitations
- Theoretical results are information-theoretic and do not link context inference to policy performance; several assumptions (beta-mixing, Lipschitz dynamics, exploratory policy) may not hold in practice.
- Evaluated only on two DMC tasks with two context variables each; absolute IQM in extrapolation remains low (0.14-0.37 in Ball-in-Cup).
Experimental setup
Datasets: DMC Ball-in-Cup (CARL contexts) DMC Walker Walk (CARL contexts)
Benchmarks: CARL Ball-in-Cup CARL Walker Walk
Metrics: IQM probability of improvement AUC
Baselines: Dreamer-DR cRSSM-S cRSSM-D
Builds on: DreamerV3 contextual MDP CARL rliable
Closest papers by meaning
- Open-World Reinforcement Learning over Long Short-Term Imagination (ICLR 2025 · similarity 0.88)
- RIG: Synergizing Reasoning and Imagination in End-to-End Generalist Policy (ICLR 2026 · similarity 0.88)
- Modeling Unseen Environments with Language-guided Composable Causal Components in Reinforcement Learning (ICLR 2025 · similarity 0.88)
- Learning Latent Dynamic Robust Representations for World Models (ICML 2024 · similarity 0.88)
- Simplifying Latent Dynamics with Softly State-Invariant World Models (NeurIPS 2024 · similarity 0.87)
- Reinforcing VLAs in Task-Agnostic World Models (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.