Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Learning Multi-Object Positional Relationships via Emergent Communication
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains emergent-communication referential-game agents to describe positional relationships between two shapes, uses a random image generator to force abstract messages, and transfers the learned language to a multi-step Object Placement MDP.
Problem
Emergent communication has focused on single objects or complex scenes; communicating positional relationships between multiple objects, and whether such emergent language generalizes to multi-step MDP tasks, is understudied.
Approach
Input variation between Speaker and Listener (separately random-generated images with same abstract relation) forces agents to emerge generalizable messages about positional relationships; the learned Speaker language is then transferred to a new Listener trained with PPO in an Object Placement grid-world task.
Key results
- Agents trained on the random dataset reach close to 80% average test accuracy on the random test set, while fixed- and variation-trained agents fail to generalize.
- In ETL image classification, the random-dataset language gives over 90% (90.8%) accuracy versus 32.8% (fixed) and 31.8% (variation).
- In Object Placement, the emergent-language Listener learns quickly and approaches the state (true relation) upper bound, outperforming raw-pixel-input and rl-scratch baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the referential-game agents with input variation between Speaker and Listener using the random dataset, rather than the fixed dataset (ablation) | Referential-game agents · Random-generator test set | representation classification accuracy: 84.4 (5.5) -> 100.0 (0.0) | Helped |
| Train the referential-game agents with input variation between Speaker and Listener using the random dataset, rather than the fixed dataset (ablation) | Referential-game agents · Random-generator test set | ETL image classification accuracy: 32.8 (8.1) -> 90.8 (2.8) | Helped |
| Train the referential-game agents with input variation between Speaker and Listener using the random dataset, rather than the variation dataset (ablation) | Referential-game agents · Random-generator test set | representation classification accuracy: 76.2 (8.9) -> 100.0 (0.0) | Helped |
| Train the referential-game agents with input variation between Speaker and Listener using the random dataset, rather than the variation dataset (ablation) | Referential-game agents · Random-generator test set | ETL image classification accuracy: 31.8 (7.2) -> 90.8 (2.8) | Helped |
| Use frozen SimCLR-pretrained image encoders instead of learning image encoders from scratch (ablation) | Referential-game agents trained with the random dataset · Random-dataset referential game | test accuracy | Hurt |
| Use emergent-language messages instead of raw-pixel target images | Object Placement Listener · Object Placement | episode reward | Helped |
| Use emergent-language messages instead of pre-trained CNN or SimCLR image features | Object Placement Listener · Object Placement | learning speed | Helped |
| Transfer the referential-game language instead of learning language from scratch in Object Placement | Object Placement agents · Object Placement | episode reward | Helped |
Surprising findings
- Pre-training the image encoder with SimCLR hurts generalization of the emergent language in the referential game.
- The variation dataset does not improve visual representation quality over the fixed dataset; input variation between Speaker and Listener is the key factor.
Limitations
- Limited to simple two-shape images with four positional relations and a simple 3x3 grid Object Placement task.
- Reliance on a random image generator for input variation may not apply in other scenarios.
- Positional relationships alone may not suffice for general tasks; generalization to out-of-distribution datasets is untested.
Experimental setup
Datasets: synthetic random shape-pair image dataset (fixed, variation, random variants)
Benchmarks: referential game test accuracy on random dataset ETL image classification Object Placement task
Metrics: test accuracy topographic similarity classification accuracy episode reward episode length
Baselines: fixed dataset variation dataset raw-pixel-input cnn-feature simclr-feature rl-scratch state
Builds on: Lazaridou et al. 2017 referential game SimCLR ETL (ease and transfer learning) REINFORCE PPO
Closest papers by meaning
- Speaking Your Language: Spatial Relationships in Interpretable Emergent Communication (NeurIPS 2024 · similarity 0.89)
- CtD: Composition through Decomposition in Emergent Communication (ICLR 2025 · similarity 0.88)
- A compressive-expressive communication framework for compositional representations (NeurIPS 2025 · similarity 0.87)
- Emergent Communication for Numerical Concepts Generalization (AAAI 2024 · similarity 0.86)
- Semantics and Spatiality of Emergent Communication (NeurIPS 2024 · similarity 0.86)
- Iterated Learning Improves Compositionality in Large Vision-Language Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.