Research map / Reinforcement learning / Visual and representation RL
FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Making
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FOUNDER, which grounds foundation-model (InternVideo2) embeddings of text or video task prompts into a DreamerV3 world model's state space and runs goal-conditioned actor-critic in imagination with a learned temporal-distance reward, evaluated on DMC, Franka Kitchen and Minecraft.
Problem
Reward-free, multi-task offline visual control where tasks are specified by text or video prompts; prior FM-to-WM grounding (GenRL) relies on step-wise visual alignment and suffers reward hacking, especially under cross-domain or cross-view video prompts.
Approach
Learn a mapping from VLM embeddings to full DreamerV3 RSSM states (deterministic plus stochastic) using paired data with a KL alignment term and an autoencoder reconstruction loss, then use a learned temporal-distance predictor to the mapped goal state as the imagination reward for goal-conditioned actor-critic learning.
Key results
- On DMC and Kitchen language tasks, FOUNDER reaches an overall normalized score of 0.81 versus 0.60 for GenRL and 0.57 for WM-CLIP.
- On cross-embodiment video tasks, FOUNDER achieves the best performance on 11 of 12 tasks and is the only method succeeding in Cheetah-based domains.
- Learned reward correlation with ground truth averages 0.54 for FOUNDER versus 0.12 for GenRL (averaged over 7 DMC tasks).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FOUNDER's grounded world-model goals and temporal-distance reward instead of GenRL's sequence matching | DMC and Kitchen tasks · DMC and Kitchen overall | normalized test performance: 0.60 -> 0.81 | Helped |
| Add temporal-distance reward to FOUNDER instead of using cosine similarity (ablation) | FOUNDER · DMC and Kitchen overall | normalized test performance: 0.52 -> 0.81 | Helped |
| Add temporal-distance reward to FOUNDER instead of using cosine similarity (ablation) | FOUNDER · Cheetah Flip | normalized test performance: -0.26 $\pm$ 0.01 -> 0.97 $\pm$ 0.02 | Helped |
| Add temporal-distance reward to FOUNDER instead of using cosine similarity (ablation) | FOUNDER · Kitchen Light | normalized test performance: 1.00 $\pm$ 0.00 -> 0.97 $\pm$ 0.18 | Hurt |
| Replace GenRL's original distance metric with temporal distance | GenRL · DMC and Kitchen tasks | normalized test performance: 0.60 -> 0.59 | No clear effect |
| Use FOUNDER's world-model grounding rather than mapping from WM states to FM representations | FOUNDER · DMC and Kitchen tasks | normalized test performance: 0.57 -> 0.81 | Helped |
| Use FOUNDER's temporal-distance reward and grounded goal states rather than GenRL | DMC reward estimation · DMC reward evaluation averaged across 7 tasks | Corr: 0.12 -> 0.54 | Helped |
| Add temporal-distance reward to FOUNDER instead of using cosine similarity (ablation) | FOUNDER reward estimation · DMC reward evaluation averaged across 7 tasks | Regret: 0.90 -> 0.07 | Helped |
Surprising findings
- Adding temporal distance to GenRL (GenRL-TempD) does not improve over GenRL, indicating the gain comes from the overall grounding architecture plus GCRL rather than the reward alone.
- FOUNDER w/o TempD (cosine reward) performs well on static tasks but fails on dynamic Walk and Run tasks, with reward hacking such as stationary agents.
Limitations
- Performance is upper-bounded by the offline environment data; prompt elements absent from the dataset may not be grounded.
- Experiments focus on short-horizon tasks; long-horizon tasks requiring decomposition are not addressed.
- Video prompts are limited to 8 frames because InternVideo2 processes at most 8 frames, making video tasks harder.
Experimental setup
Datasets: DMC offline dataset from GenRL (Plan2Explore and task buffers) Franka Kitchen offline dataset Quadruped dataset (self-collected) Minecraft MineDojo offline data
Benchmarks: DeepMind Control Suite (Cheetah, Walker, Stickman, Quadruped) Franka Kitchen Minecraft (MineDojo tasks)
Metrics: normalized return success rate RankCorr Regret@1 Precision Recall F1
Baselines: GenRL WM-CLIP GenRL-TempD FOUNDER w/o TempD TD3 HILP MineCLIP-IQL
Builds on: DreamerV3 GenRL Plan2Explore MineCLIP
Closest papers by meaning
- Self-Improving Embodied Foundation Models (NeurIPS 2025 · similarity 0.87)
- Grounding Video Models to Actions through Goal Conditioned Exploration (ICLR 2025 · similarity 0.86)
- DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning (ICML 2025 · similarity 0.86)
- Neural Motion Simulator Pushing the Limit of World Models in Reinforcement Learning (CVPR 2025 · similarity 0.86)
- Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models (NeurIPS 2026 · similarity 0.86)
- HarmonyDream: Task Harmonization Inside World Models (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.