Research map / Image generation / Text-to-image diffusion
Moonshine: Distilling Game Content Generators into Steerable Generative Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Distills the Brogue constructive dungeon generator into text-conditioned PCGML models (Five-Dollar Model and a discrete diffusion model) using LLM-labeled synthetic map-description pairs.
Problem
Procedural content generation via machine learning lacks controllability and labeled training data; text-conditioned map generation needs large richly labeled datasets that are costly to produce by hand.
Approach
Generate map data with the Brogue constructive PCG algorithm, compute metadata heuristics about rooms and tiles, and have GPT-4 Turbo write text labels from few-shot human examples; then train text-conditioned generators (FDM and a conditional discrete diffusion model) on these pairs so they replicate the algorithm while accepting plain-text control (Text-to-game-Map task).
Key results
- Long LLM descriptions score higher than short ones within generated sets (e.g., METEOR 19.50 vs 12.64; SPICE 11.31 vs 3.89) and against 30 human references (BLEU-1 70.84 vs 50.58).
- DDM produces more diverse maps than FDM, with fewer disconnected components and lower fragmentation in connectivity analysis.
- FDM achieves slightly higher overall description-alignment scores than DDM but overfits after about 20 epochs and produces near-uniform outputs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use long rather than short synthetic descriptions | LLM-generated map descriptions · Generated descriptions, within-description comparison | BLEU_1: 54.71 -> 58.37 | Hurt |
| Use long rather than short synthetic descriptions | LLM-generated map descriptions · Generated descriptions, within-description comparison | BLEU_2: 26.57 -> 26.56 | No clear effect |
| Use long rather than short synthetic descriptions | LLM-generated map descriptions · Generated descriptions, within-description comparison | SPICE: 03.89 -> 11.31 | Helped |
| Use long rather than short synthetic descriptions | LLM-generated map descriptions · Generated descriptions vs human references | BLEU_1: 50.58 -> 70.84 | Helped |
| Use long rather than short synthetic descriptions | LLM-generated map descriptions · Generated descriptions vs human references | SPICE: 00.77 -> 03.43 | Helped |
| Use the Discrete Diffusion Model rather than the Five-Dollar Model | Moonshine text-to-map generation · Brogue map generation from text prompts | diversity | Helped |
| Use the Five-Dollar Model rather than the Discrete Diffusion Model | Moonshine text-to-map generation · Brogue map generation from text prompts | overall scores | Helped |
| Use the Five-Dollar Model rather than the Discrete Diffusion Model | Moonshine text-to-map generation · Brogue map generation from text prompts | output diversity | Hurt |
Surprising findings
- FDM overfits after about 20 epochs, learning only training mappings, while DDM generalizes better to diverse unseen outputs.
- Short descriptions correspond more closely with map images under frozen pretrained CLIP, while long descriptions fit better after CLIP fine-tuning.
Limitations
- Loss of fine-grained control, partly due to LLM annotation quality and limited tile types (14 terrain tiles).
- DDM is limited by training dataset size; generated regions are not always connected.
- Only 3,000 maps were labeled with GPT-4 Turbo due to financial constraints; no human study of designer use.
Experimental setup
Datasets: Brogue generated dungeon maps LLM-generated map descriptions human-written map descriptions (few-shot examples and 30 references)
Benchmarks: Brogue ground-truth maps
Metrics: BLEU ROUGE-L METEOR SPICE CLIP score connected component analysis (disconnected components, fragm reconstruction MSE
Baselines: Five-Dollar Model Brogue constructive algorithm
Builds on: Five-Dollar Model denoising diffusion probabilistic models gte-large-en-v1.5
Closest papers by meaning
- Procedural Level Generation with Diffusion Models from a Single Example (AAAI 2024 · similarity 0.87)
- GameGen-X: Interactive Open-world Game Video Generation (ICLR 2025 · similarity 0.86)
- Diffusion Models Are Real-Time Game Engines (ICLR 2025 · similarity 0.84)
- Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (ICML 2024 · similarity 0.82)
- Unbounded: A Generative Infinite Game of Character Life Simulation (ICLR 2025 · similarity 0.82)
- GAVEL: Generating Games via Evolution and Language Models (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.