Research map / Reinforcement learning / Visual and representation RL
ReCoRe: Regularized Contrastive Representation Learning of World Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ReCoRe, a DreamerV2-based world model that combines InfoNCE contrastive representation learning with an intervention-invariant auxiliary regularizer (depth prediction, or segmentation/denoising) to improve sample efficiency and OoD generalization for visual navigation and DMControl.
Problem
Model-based RL for visual navigation generalizes poorly under appearance variations and is sample-inefficient; naive contrastive learning inside world models collapses because the world model and policy are optimized separately.
Approach
Combine contrastive (InfoNCE) invariant representation learning with a world model, and stabilize it by adding an auxiliary intervention-invariant task (e.g., depth prediction during training only) that forces the latent state to predict geometry consistently under augmentations, preventing feature collapse.
Key results
- On iGibson OoD PointGoal at 500k steps, ReCoRe reaches 77.3 SR / 0.65 SPL on held-out scenes (env avg 59.7 SPL), outperforming DV2+G DINO (65.8 SR) and CURL (41.9 SR).
- Removing the depth regularizer collapses performance (ReCoRe - D: 1.2 SR at 500k vs 77.3 for ReCoRe), while naive contrastive learning fails in model-based RL.
- On Gibson sim-to-real at 500k steps, ReCoRe achieves 71.9 average SR vs 60.3 for DV2+G DINO and 36.7 for CURL; on DMControl it wins 8 of 12 settings against CURL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ReCoRe instead of DV2+G DINO | iGibson 1.0 PointGoal navigation · iGibson 1.0 PointGoal, Env Avg, 500k | SR: 50.0 -> 59.7 | Helped |
| Use ReCoRe instead of DV2+G DINO | Gibson sim-to-real PointGoal navigation · Gibson sim-to-real PointGoal, Env Avg, 500k | SR: 60.3 -> 71.9 | Helped |
| Add semantic segmentation as an auxiliary regularizer | ReCoRe on DMControl · DMControl suite, 8 out of 12 experiments | total rewards | Mixed |
| Add contrastive learning (ablation) | ReCoRe on iGibson 1.0 PointGoal · iGibson 1.0 PointGoal, Env Avg, 500k | SR: 5.0 -> 59.7 | Helped |
| Add data augmentation without contrastive learning (ablation) | ReCoRe on iGibson 1.0 PointGoal · iGibson 1.0 PointGoal, Env Avg, 500k | SR: 5.0 -> 22.1 | Helped |
| Add depth auxiliary reconstruction (ablation) | ReCoRe on iGibson 1.0 PointGoal · iGibson 1.0 PointGoal, Env Avg, 500k | SR: 0.8 -> 59.7 | Helped |
| Replace depth reconstruction with RGB image reconstruction (ablation) | ReCoRe on iGibson 1.0 PointGoal · iGibson 1.0 PointGoal, Env Avg, 500k | SR: 59.7 -> 19.2 | Hurt |
| Use semantic segmentation instead of image denoising as the auxiliary task (ablation) | ReCoRe on DMControl · DMControl suite | total rewards | Helped |
Surprising findings
- Naive integration of contrastive loss into a world model collapses (ReCoRe - D under 1% success rate in model-based RL).
- Data augmentation alone improves DreamerV2 for OoD generalization, which had previously been shown mainly for model-free RL.
Limitations
- Depth regularizer requires depth supervision during training, limiting use where depth is unavailable (denoising is not truly invariant and gives limited gains on DMControl).
- Training cost roughly twice DreamerV2; 100k-step ReCoRe is only on par with DV2+G DINO on iGibson OoD.
- Hyperparameters tuned per task were not used for DMControl; a single shared parameter set was used.
Experimental setup
Datasets: iGibson 1.0 Gibson DeepMind Control Suite
Benchmarks: iGibson 1.0 PointGoal navigation Gibson sim-to-real DeepMind Control Suite
Metrics: success rate SPL
Baselines: RAD CURL DreamerV2 DreamerV2 + DA Masked World Models DV2+G DINO SAC+AE Pixel SAC
Builds on: DreamerV2 InfoNCE MoCo CURL Masked World Models
Closest papers by meaning
- DyMoDreamer: World Modeling with Dynamic Modulation (NeurIPS 2025 · similarity 0.87)
- Zero-shot World Models via Search in Memory (NeurIPS 2025 · similarity 0.87)
- RLVR-World: Training World Models with Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
- Learning Latent Dynamic Robust Representations for World Models (ICML 2024 · similarity 0.86)
- Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining (ICLR 2025 · similarity 0.86)
- EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modeling (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.