Research map / Learning theory and algorithms / Submodular and game algorithms
Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reinterprets Lewis's signaling game as a beta-VAE with a learnable receiver-side language-model prior, showing that prior choice affects Zipf's law of abbreviation and Harris's articulation scheme in emergent languages.
Problem
Conventional signaling-game objectives have implicit, unnatural message priors, which has been reported to prevent emergent languages from following Zipf's law of abbreviation and producing meaningful word segments (Harris's articulation scheme).
Approach
Treat the receiver as a generative model with an explicit prior over messages (a GRU language model), and optimize the signaling game as the ELBO of a beta-VAE; the prior acts as a surprisal term that encourages natural segmentation, unlike implicit uniform or length-penalty priors.
Key results
- The ELBO objective with a learnable GRU prior makes the number of boundaries per message increase monotonically with the number of attributes, satisfying criterion C1 only for the proposed objective.
- The number of distinct segments decreases monotonically with the number of values only for the proposed objective, satisfying criterion C2.
- Both C-TopSim and W-TopSim improve with the proposed objective, although delta TopSim remains negative for all objectives.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the ELBO objective with a learnable neural language-model prior instead of the conventional objective or fixed length-based prior | Lewis's signaling game · Attribute-value objects; variable-length messages | boundaries per message | Helped |
| Use the ELBO objective with a learnable neural language-model prior instead of the conventional objective or fixed length-based prior | Lewis's signaling game · Attribute-value objects; variable-length messages | distinct segments | Helped |
| Use the ELBO objective with a learnable neural language-model prior instead of the conventional objective or fixed length-based prior | Lewis's signaling game · Attribute-value objects; variable-length messages | W-TopSim | Helped |
| Use the ELBO objective with a learnable neural language-model prior instead of the conventional objective or fixed length-based prior | Lewis's signaling game · Attribute-value objects; variable-length messages | C-TopSim | Helped |
| Use the ELBO objective with a learnable neural language-model prior instead of the conventional objective or fixed length-based prior | Lewis's signaling game · Attribute-value objects; variable-length messages | deltaTopSim | No clear effect |
| Use the ELBO objective with a learnable neural language-model prior instead of the conventional objective or fixed length-based prior | Lewis's signaling game · Attribute-value objects; moderate alphabet sizes | Zipf's law of abbreviation tendency | Helped |
Surprising findings
- delta TopSim stays negative even with the proposed objective, so criterion C3 is not met, which the authors attribute possibly to spelling variation of segments.
Limitations
- Only reconstruction games are studied; discrimination and populated (multi-agent) games are left to future work.
- Only GRU is used as the prior language model; cognitively motivated LMs are not tested.
- The mechanism behind improved segmentation is hypothesized rather than proven mathematically.
Experimental setup
Datasets: attribute-value object space with 4096 objects (power-law object distribution)
Benchmarks: emergent language segmentation criteria C1-C3 (boundaries per message, distinct segments, TopSim differences)
Metrics: number of boundaries per message number of distinct segments delta TopSim C-TopSim W-TopSim
Baselines: conventional signaling game objective with entropy regularizer ELBO with length-penalty prior (monkey typing prior)
Builds on: beta-VAE Lewis's signaling game REWO Harris's articulation scheme surprisal theory
Closest papers by meaning
- Unsupervised Translation of Emergent Communication (AAAI 2025 · similarity 0.87)
- A compressive-expressive communication framework for compositional representations (NeurIPS 2025 · similarity 0.87)
- Bridging semantics and pragmatics in information-theoretic emergent communication (NeurIPS 2024 · similarity 0.87)
- Semantics and Spatiality of Emergent Communication (NeurIPS 2024 · similarity 0.87)
- AlphaZero Neural Scaling and Zipf's Law: a Tale of Board Games and Power Laws (NeurIPS 2025 · similarity 0.86)
- Emergent Communication for Numerical Concepts Generalization (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.