Research map / Multi-agent systems and games / Human-AI and goal-conditioned
GenEx: Generating an Explorable World
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GenEx, a system that generates a 360° explorable panoramic world from a single image via panoramic video diffusion trained on physics-engine data, and uses the imagined observations to improve embodied decision-making with GPT-4o and humans.
Problem
Enabling AI agents to explore and reason about unseen parts of a 3D physical environment from only a single initial view, with coherent, 3D-consistent generated observations.
Approach
Model world exploration as action-conditioned panoramic video generation (image-to-panorama initialization followed by rotation-plus-distance-conditioned transitions with spherical-consistent learning), and use the imagined panoramas as extra observations for an imagination-augmented embodied policy.
Key results
- GenEx reaches FVD 69.5, PSNR 30.2 and SSIM 0.94 on video generation, beating the 6-view cubemap baseline (FVD 196.7, PSNR 26.1).
- GPT-4o with GenEx imagined observations reaches 85.22% accuracy on the single-agent EQA evaluation versus 46.10% for multimodal GPT-4o.
- On the multi-agent evaluation, GPT-4o with GenEx reaches 94.87% accuracy versus 21.88% for multimodal GPT-4o.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add spherical-consistency learning (SCL) (ablation) | GenEx · Video generation quality | FVD: 81.9 -> 69.5 | Helped |
| Add spherical-consistency learning (SCL) (ablation) | GenEx · Video generation quality | MSE: 0.05 -> 0.04 | Helped |
| Add spherical-consistency learning (SCL) (ablation) | GenEx · Video generation quality | LPIPS: 0.05 -> 0.03 | Helped |
| Add spherical-consistency learning (SCL) (ablation) | GenEx · Video generation quality | PSNR: 29.4 -> 30.2 | Helped |
| Add spherical-consistency learning (SCL) (ablation) | GenEx · Video generation quality | SSIM: 0.91 -> 0.94 | Helped |
| Provide imagined observations generated by GenEx rather than only an image | Human policy · Embodied decision-making | accuracy: 91.50 -> 94.00 | Helped |
| Provide imagined observations generated by GenEx rather than only a visual input | GPT-4o policy · Embodied decision-making | accuracy: 46.10 -> 85.22 | Helped |
| Provide imagined observations generated by GenEx rather than only an image | Human policy · Multi-agent embodied decision-making | accuracy: 55.24 -> 77.41 | Helped |
| Provide imagined observations generated by GenEx rather than only a visual input | GPT-4o policy · Multi-agent embodied decision-making | accuracy: 21.88 -> 94.87 | Helped |
Surprising findings
- Unimodal GPT-4o and Gemini-1.5 (text only) often outperform their multimodal counterparts, which the authors attribute to vision without imagination being misleading.
- Humans given GenEx-imagined videos outperform humans given the real image alone.
Limitations
- Relies on physics engine (simulated) training data; sim-to-real gap is acknowledged as a challenge.
- Generation quality can degrade over long exploration, requiring GPT-4o action selection to avoid model collapse.
- Evaluation of embodied decision making uses an extended EQA setup and human participants with limited described scale.
Experimental setup
Datasets: Unreal Engine 5 city assets exploration trajectories Unity animated world exploration trajectories
Benchmarks: GenEx-EQA (extended) Multi-agent EQA (extended)
Metrics: FVD MSE LPIPS PSNR SSIM accuracy confidence logic accuracy latent MSE (IELC)
Baselines: Gemini-1.5 GPT-4o 6-view cubemap baseline GenEx w/o SCL
Builds on: FLUX.1 spherical-consistent learning DUSt3R
Closest papers by meaning
- Exploration-Driven Generative Interactive Environments (CVPR 2025 · similarity 0.87)
- SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning (NeurIPS 2026 · similarity 0.86)
- Genie: Generative Interactive Environments (ICML 2024 · similarity 0.86)
- Imagined Autocurricula (NeurIPS 2025 · similarity 0.84)
- Learning 3D Persistent Embodied World Models (NeurIPS 2025 · similarity 0.84)
- 3D-VLA: A 3D Vision-Language-Action Generative World Model (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.