Research map / Vision-language and multimodal / Robot action and navigation
GenRL: Multimodal-foundation world models for generalization in embodied agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GenRL, which aligns a frozen video-language model's embedding space with a generative world model's latent space using only vision data, enabling language- and video-prompted policy learning in imagination, including a data-free variant.
Problem
Generalist embodied agents need to solve many tasks specified by language or vision without reward engineering, but VLM rewards require fine-tuning and multimodal embodied data is scarce.
Approach
Connect a VLM's embedding space to a world model's latent space via a latent connector trained on visual embeddings, plus an aligner network trained with noise-perturbed vision embeddings so language embeddings land near their visual counterparts, requiring no language annotations; policies are then trained in imagination with a cosine reward between latent targets and imagined states.
Key results
- GenRL reaches 0.80 overall normalized reward on language-to-action in-distribution tasks with video-language VLM, versus 0.41 for TD3 and 0.70 for WM-CLIP.
- In the aligner ablation, GenRL with aligner gets 0.76 overall versus 0.17 without the aligner.
- On kitchen tasks with LIV representation, GenRL reaches 0.76 overall versus 0.20 for the best LIV baseline (IQL + LIV).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GenRL instead of the strongest reported baseline, WM-CLIP (video-language) | offline RL from language prompts · In-distribution tasks, overall | normalized episodic reward: 0.70 ± -> 0.80 ± | Helped |
| Use GenRL instead of the strongest reported LIV baseline, IQL + LIV | Kitchen tasks · Kitchen tasks, overall | normalized episodic reward: 0.20 ± -> 0.76 ± | Helped |
| Add the aligner before feeding language embeddings to the connector (ablation) | GenRL · Overall | normalized episodic reward: 0.17 ± -> 0.76 ± | Helped |
| Add the aligner before feeding language embeddings to the connector (ablation) | GenRL · Cheetah | normalized episodic reward: 0.32 ± -> 0.93 ± | Helped |
| Add the aligner before feeding language embeddings to the connector (ablation) | GenRL · Stickman | normalized episodic reward: 0.09 ± -> 0.66 ± | Helped |
| Add the aligner before feeding language embeddings to the connector (ablation) | GenRL · Walker | normalized episodic reward: 0.19 ± -> 0.75 ± | Helped |
| Add the aligner before feeding language embeddings to the connector (ablation) | GenRL · Kitchen | normalized episodic reward: 0.25 ± -> 0.76 ± | Helped |
| Add the aligner before feeding language embeddings to the connector (ablation) | GenRL · Quadruped | normalized episodic reward: 0.17 ± -> 0.91 ± | Helped |
| Process the prompt embedding with GenRL's aligner before computing cosine-similarity rewards (ablation) | WM-CLIP-V and TD3-V · Overall | normalized episodic reward: WM-CLIP-V: 0.67 ± -> 0.68 ±; TD3-V: 0.40 ± -> 0.42 ± | No clear effect |
| Learn the policy in imagination without dataset access | GenRL · Data-free policy learning | normalized episodic reward | Hurt |
| Use the complete, varied training dataset rather than a restricted subset | GenRL data-free policy learning · Walker training-data distribution | task performance | Helped |
| Train the world model on stand-only data rather than a diverse dataset | GenRL data-free policy learning · Walker training-data distribution | task performance | Hurt |
Surprising findings
- Data-free policy learning shows only a slight performance drop relative to offline GenRL and even performs better in the kitchen domain.
- Static kitchen tasks are sometimes won by baselines, because GenRL's inferred target sequences are slightly in motion even for static prompts.
Limitations
- Reliance on reconstruction-based world model causes blurry targets in complex observations such as Minecraft.
- Out-of-distribution manipulation tasks in the kitchen domain are hard; agent struggles beyond dataset behaviors.
- Static tasks suffer because target sequences are inferred as moving, and the VLM has limited temporal span (8 frames).
Experimental setup
Datasets: walker, cheetah, quadruped, stickman, kitchen vision datasets (Plan2Explore exploration and DreamerV3 expert data) minecraft small dataset
Benchmarks: dm_control walker, cheetah, quadruped, stickman (new) Franka kitchen
Metrics: min-max normalized episodic reward (random=0, expert=1)
Baselines: IQL TD3+BC TD3 WM-CLIP SigLIP image-language reward LIV
Builds on: DreamerV3 InternVideo2 SigLIP LIV
Closest papers by meaning
- ModularAgent: A Task-Aware Modular Framework for Joint Optimization of Multimodal Large Language Models and World Models (CVPR 2026 · similarity 0.91)
- From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons (CVPR 2025 · similarity 0.90)
- Large Language Models as Generalizable Policies for Embodied Tasks (ICLR 2024 · similarity 0.89)
- EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (ICML 2025 · similarity 0.89)
- 3D-VLA: A 3D Vision-Language-Action Generative World Model (ICML 2024 · similarity 0.89)
- EVLP: Learning Unified Embodied Vision-Language Planner with Reinforced Supervised Fine-Tuning (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.