Research map / Large language models / Agents, code and math
GAVEL: Generating Games via Evolution and Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GAVEL, which fine-tunes CodeLlama-13b as a fill-in-the-middle mutation operator over Ludii game descriptions and runs MAP-Elites over concept-based PCA archives to generate playable novel board games.
Problem
Automatically generating novel, playable and interesting board games represented in a large, expressive game description language (Ludii), where search space is huge and originality/quality evaluation is hard.
Approach
Fine-tune a code LLM with fill-in-the-middle objective on Ludii game code to act as a mutation operator inside a MAP-Elites quality-diversity search, using a game-agnostic hierarchical fitness from simulated playouts and an archive built from PCA-reduced Ludii concept vectors.
Key results
- GAVEL achieves QD score 395.62 vs 296.92 for pure sampling, 268.16 for GPT-4o, and 341.17 for GAVEL-UCB (3 runs each).
- Over 500 generations from 14 seeds, GAVEL produced 185 novel variations, 130 playable, with 62 occupying cells not covered by any Ludii game.
- Fitness-based archive filling: GAVEL fills about 106.67 cells with fitness>0.5 vs 83.00 for pure sampling and 80.67 for GPT-4o.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MAP-Elites search rather than pure sampling | GAVEL with fine-tuned CodeLlama-13B · MAP-Elites archive | QD score: 296.92 { ± 14.84} -> 395.62 { ± 17.46} | Helped |
| use GAVEL instead of direct GPT-4o sampling | game generation · MAP-Elites archive | QD score: 268.16 { ± 17.33} -> 395.62 { ± 17.46} | Helped |
| select mutation locations with UCB (ablation) | GAVEL · MAP-Elites archive | QD score: 395.62 { ± 17.46} -> 341.17 { ± 14.39} | Hurt |
| select mutation locations with UCB (ablation) | GAVEL · MAP-Elites archive | # playable: 117.67 { ± 9.46} -> 96.67 { ± 6.02} | No clear effect |
| mutate validation-set games rather than training-set games (ablation) | fine-tuned CodeLlama-13b · game mutation, Novel & Valid | proportion: 16.67 -> 46.00 | Helped |
| increase sampling temperature from 0.5 to 1.5 (ablation) | training-set game mutations, Top-K 20 · game mutation | Novel & Valid: 16.67 -> 22.33 | Helped |
| increase sampling temperature from 0.5 to 1.5 (ablation) | validation-set game mutations, Top-K 20 · game mutation | Novel & Valid: 46.00 -> 33.33 | Hurt |
| increase archive dimensionality at fixed target size (ablation) | MAP-Elites archive · Ludii training games | unique occupied cells | Hurt |
Surprising findings
- Mutations of training-set games are mostly exact duplicates of the original, so held-out games were used for initialization to avoid memorization.
- CVT archives on the full 510-dimensional concept space collapsed games into very few cells, so PCA was used instead.
Limitations
- Many generated games contain unused rules that the evaluation cannot detect.
- Fitness metrics are only minimal proxies of interestingness; human expert filtering was needed.
- Held-out initialization games share features, limiting archive diversity; statistical significance is limited with three runs.
Experimental setup
Datasets: Ludii game library (1182 games; 574 used for training, 14 held out)
Benchmarks: Ludii concept-based archive QD score playability and fitness metrics from simulated playouts
Metrics: QD score number of playable cells number of cells with fitness>0.5 novel cells balance decisiveness completion agency coverage strategic depth
Baselines: GAVEL-UCB Pure Sampling GPT-4o
Builds on: evolution through large models MAP-Elites CodeLlama LoRA Ludii
Closest papers by meaning
- Mastering Board Games by External and Internal Planning with Language Models (ICML 2025 · similarity 0.89)
- Evaluating LLMs in Open-Source Games (NeurIPS 2025 · similarity 0.86)
- GTBench: Uncovering the Strategic Reasoning Capabilities of LLMs via Game-Theoretic Evaluations (NeurIPS 2024 · similarity 0.86)
- Can Large Language Models Master Complex Card Games? (NeurIPS 2025 · similarity 0.85)
- PokerBench: Training Large Language Models to Become Professional Poker Players (AAAI 2025 · similarity 0.84)
- Generating Creative Chess Puzzles (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.