Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Exploration-Driven Generative Interactive Environments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a framework that pretrains an open Genie reimplementation (GenieRedux-G) on random-agent interaction data from 200+50 retro games (RetroAct) and then fine-tunes it on data from an uncertainty-driven AutoExplore agent for new environments.
Problem
World models like Genie need costly action-labeled video demonstrations; random-agent data is limited in exploration, so visual fidelity and controllability suffer in new environments.
Approach
Train an environment-independent exploration agent whose reward is the entropy of the world model's token predictions (uncertainty), so it collects diverse interaction data that fine-tunes a multi-environment world model without any environment reward; also introduces a Token Distance Cross-Entropy loss and a token skip connection.
Key results
- Fine-tuning on AutoExplore data improves PSNR by up to 7.4 over random-agent fine-tuning and controllability up to 1.4 delta PSNR (Adventure Island II autoregressive: 2.09 vs 1.24).
- GenieRedux-G-50 reaches 26.36 PSNR and 0.450 delta PSNR on 50 environments after random-agent pretraining.
- In user studies (19 participants), the AutoExplore-trained model is rated closer to ground truth than the random-agent model on Super Mario Bros. and Adventure Island II.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace LAM-predicted actions with agent-provided actions for Dynamics conditioning | GenieRedux · CoinRun Basic Test Set | PSNR: 25.51 -> 33.41 | Helped |
| Replace LAM-predicted actions with agent-provided actions for Dynamics conditioning | GenieRedux · CoinRun Diverse Test Set | PSNR: 23.82 -> 31.66 | Helped |
| Train with diverse data collected by a trained agent instead of random-agent data | GenieRedux-G · CoinRun Diverse Test Set | PSNR: 31.66 -> 34.44 | Helped |
| Add token input skip connection (ablation) | GenieRedux-G-50 · Platformers-200/50 test set | PSNR: 26.36 -> 26.65 | Helped |
| Add Token Distance Cross-Entropy Loss (ablation) | GenieRedux-G-50 + Token Input · Platformers-200/50 test set | PSNR: 26.65 -> 27.06 | Helped |
| Fine-tune on AutoExplore Agent data rather than random-agent data | GenieRedux-G-50 · Adventure Island II | PSNR: 27.04 -> 30.60 | Helped |
| Fine-tune on AutoExplore Agent data rather than random-agent data | GenieRedux-G-50 · Super Mario Bros | PSNR: 34.54 -> 36.13 | Helped |
| Fine-tune on AutoExplore Agent data rather than random-agent data | GenieRedux-G-50 · Smurfs | PSNR: 21.83 -> 27.45 | Helped |
| Fine-tune on AutoExplore Agent data rather than random-agent data | GenieRedux-G-50 · Adventure Island II | ΔPSNR: 1.19 -> 1.47 | Helped |
| Fine-tune on AutoExplore Agent data rather than random-agent data | GenieRedux-G-50 · Super Mario Bros | ΔPSNR: 0.54 -> 0.57 | Helped |
| Fine-tune on AutoExplore Agent data rather than random-agent data | GenieRedux-G-50 · Smurfs | ΔPSNR: 0.65 -> 1.55 | Helped |
| Use AutoExplore Agent data instead of random-agent data for autoregressive fine-tuning | GenieRedux-G-50-ft · Adventure Island II | PSNR: 27.82 -> 33.61 | Helped |
Surprising findings
- Exploration-based fine-tuning on three games gives gains even where random-agent fine-tuning yields little change, e.g. Super Mario Bros. delta PSNR of the random-fine-tuned model barely moved (0.56 to 0.54).
Limitations
- Experiments use low resolution (64x64 world model, upscaled to 256x256) and short sequences of 16 frames.
- Evaluated on only three exploration environments plus a small set of pretraining environments.
- Controllability gains are modest for some games, and the uncertainty reward is tied to the world model's own classification entropy.
Experimental setup
Datasets: RetroAct Platformers-200 Platformers-50 CoinRun (Procgen) Stable Retro environments
Benchmarks: CoinRun test sets Platformers-50 test set Super Mario Bros. exploration test Adventure Island II exploration test Smurfs exploration test
Metrics: FID PSNR SSIM delta PSNR user rating
Baselines: GenieRedux GenieRedux-G-TA Random agent data RF-based SSE exploration agent VAE-based SSE exploration agent
Builds on: Genie MaskGIT Plan2Explore Stable Retro IMPALA-style CNN-LSTM actor-critic
Closest papers by meaning
- Genie: Generative Interactive Environments (ICML 2024 · similarity 0.92)
- SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning (NeurIPS 2026 · similarity 0.89)
- Learning Interactive Real-World Simulators (ICLR 2024 · similarity 0.88)
- Imagined Autocurricula (NeurIPS 2025 · similarity 0.87)
- Behavioral Exploration: Learning to Explore via In-Context Adaptation (ICML 2025 · similarity 0.87)
- GenEx: Generating an Explorable World (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.