Research map / Large language models / Agents, code and math
Emergent Communication for Numerical Concepts Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces NumGame, a referential-game environment with a NumWorld dataset, and a two-stage training method (NumSen pretraining then NumRel arithmetic training) so emergent language generalizes counting and calculation to unseen quantities.
Problem
Emergent communication research has focused on qualitative object attributes, not numerical concepts; it is unclear how agents can count and perform arithmetic on unseen quantities through emergent language.
Approach
Combines a referential game with a number-sense pretraining stage (subitizing and magnitude estimation) and an arithmetic-relation training stage (add, sub, max over quantities), using soft ordinal labels so the emergent messages capture order relations between numbers.
Key results
- Full two-stage method (NumSenRel) reaches 0.97 accuracy and 0.86 AMI on unseen-quantity counting, versus 0.71 accuracy for the Base method.
- On unseen-quantity calculating, NumSenRel reaches 0.83 accuracy and 0.83 AMI, versus 0.18 accuracy for Base and 0.27 without training.
- Removing NumRel drops counting accuracy to about 0.7, indicating arithmetic training is the main driver of generalization.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add NumRel calculation training (ablation) | Base counting method · OOD counting | accuracy: 0.71 -> 0.94 | Helped |
| Add NumSen pretraining (ablation) | Base counting method · OOD counting | accuracy: 0.71 -> 0.72 | No clear effect |
| Add NumSen pretraining (ablation) | NumRel counting method · OOD counting | accuracy: 0.94 -> 0.97 | Helped |
| Add NumSen pretraining (ablation) | Base counting method · OOD counting | adjusted mutual information (AMI): 0.70 -> 0.76 | Helped |
| Add NumSen pretraining (ablation) | NumRel counting method · OOD counting | adjusted mutual information (AMI): 0.82 -> 0.86 | Helped |
| Add NumRel calculation training (ablation) | Base calculating method · OOD calculating | accuracy: 0.18 -> 0.79 | Helped |
| Add NumSen pretraining (ablation) | NumRel calculating method · OOD calculating | accuracy: 0.79 -> 0.83 | Helped |
| Add NumSen pretraining (ablation) | Base calculating method · OOD calculating | accuracy: 0.18 -> 0.20 | Helped |
| Add NumSen pretraining (ablation) | Base calculating method · OOD calculating | adjusted mutual information (AMI): 0.64 -> 0.70 | Helped |
| Add NumSen pretraining (ablation) | NumRel calculating method · OOD calculating | adjusted mutual information (AMI): 0.81 -> 0.83 | Helped |
| Add NumSen pretraining (ablation) | NoRel training method · Lang validation | accuracy: 0.82 -> 0.84 | Helped |
| Add NumSen pretraining (ablation) | NumRel training method · Lang validation | accuracy: 0.78 -> 0.81 | Helped |
Surprising findings
- The Base training method yields lower calculating accuracy (0.18) than no training at all (0.27), suggesting it overfits individual numbers.
- NumSen pretraining mainly improves semantic stability (AMI) rather than accuracy.
Limitations
- Only addition, subtraction, and max are studied; more complex calculations are left for future work.
- Calculating accuracy (about 0.8) remains below counting accuracy.
- Experiments use a synthetic ShapeWorld-based image environment with few-shot OOD adaptation.
Experimental setup
Datasets: NumWorld (built on ShapeWorld) ShapeWorld
Benchmarks: NumWorld OOD counting test NumWorld OOD calculating test
Metrics: accuracy adjusted mutual information (AMI)
Baselines: Base (single-quantity training) NoRel w/o training
Builds on: referential game Gumbel-Softmax ShapeWorld EGG toolkit
Closest papers by meaning
- Learning Multi-Object Positional Relationships via Emergent Communication (AAAI 2024 · similarity 0.86)
- Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments (ICLR 2024 · similarity 0.86)
- Atomic Thinking of LLMs: Decoupling and Exploring Mathematical Reasoning Abilities (NeurIPS 2025 · similarity 0.86)
- A compressive-expressive communication framework for compositional representations (NeurIPS 2025 · similarity 0.85)
- CogMath: Assessing LLMs' Authentic Mathematical Ability from a Human Cognitive Perspective (ICML 2025 · similarity 0.85)
- A Matter of Interest: Understanding Interestingness Judgments of Math Problems in Humans and Language Models (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.