Research map / Reinforcement learning / Visual and representation RL

Zero-shot World Models via Search in Memory

NeurIPS 2025 · Reinforcement learning › Visual and representation RL · approach: Robust visual encoders for RL · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes zero-shot world models that approximate latent transition dynamics via similarity search over stored VAE latents (Rollout, Replay-L2, Replay-KL), without training a dynamics model, and compares them to PlaNet.

Problem

Learning-based world models need large data and compute to model environment dynamics; the paper asks whether next-state and long-horizon dynamics can be predicted without training a dynamics model.

Approach

Combine similarity search over a memory of encoded VAE latent transitions with the stochastic (Normal) latent representation to sample next latents, iterating retrieve-and-reconstruct for long horizons, with three memory structures (rollout buffer, replay buffer with L2 or KL-divergence retrieval).

Key results

• On SuperTuxKart planning, Replay-KL reaches 15.66 mean reward on snowmountain versus 5.32 for PlaNet, and replay-based methods win 4 of 5 tracks.
• Search-based Rollout and Replay-KL are visually comparable to PlaNet in one-step prediction and better in long-horizon prediction on SuperTuxKart; Replay-L2 hallucinates in reconstruction.
• Inference time for the search-based methods is about 0.0008 to 0.0025 s versus 0.044 s for PlaNet on SuperTuxKart.

Measured effects reported in the paper

Surprising findings

• Replay-L2 achieves low latent KL error yet produces visually incoherent decoded images, showing latent-space metrics can mislead.
• Increasing the number of stored trajectories substantially improves the PlaNet baseline but only marginally helps the search-based models.

Limitations

• Performance depends on latent-space coverage of stored trajectories, so it struggles in open-ended tasks such as Minecraft Navigate.
• Requires access to future actions for prediction, so it is not directly usable for action selection without extra assumptions.
• Replay-L2 and some methods produce visually incoherent or hallucinated reconstructions; memory grows with stored data (about 12GB RAM).

Experimental setup

Datasets: SuperTuxKart MineRL Treechop-v0 MineRL Navigate-v0 Atari Seaquest Atari Space Invaders

Benchmarks: SuperTuxKart MineRL Treechop-v0 MineRL Navigate-v0 Atari Seaquest Atari Space Invaders

Metrics: KL divergence L1 distance SSIM mean reward inference time

Baselines: PlaNet

Builds on: PlaNet Dreamer locally weighted learning Zero-shot Imitation Policy beta-VAE

Closest papers by meaning

• Learning Visual Feature-Based World Models via Residual Latent Action (NeurIPS 2026 · similarity 0.90)
• DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning (ICML 2025 · similarity 0.89)
• Reinforcing VLAs in Task-Agnostic World Models (NeurIPS 2026 · similarity 0.89)
• RLVR-World: Training World Models with Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• Open-World Reinforcement Learning over Long Short-Term Imagination (ICLR 2025 · similarity 0.89)
• Learning Latent Dynamic Robust Representations for World Models (ICML 2024 · similarity 0.89)