Research map / Multi-agent systems and games / Human-AI and goal-conditioned
CtD: Composition through Decomposition in Emergent Communication
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Composition through Decomposition (CtD), a two-step emergent communication method where agents first learn a VQ codebook of basic concepts via multi-target games, then compose them into phrases for unseen images, sometimes zero-shot.
Problem
Neural emergent communication agents typically fail to develop compositional language that describes novel combinations of known concepts in images.
Approach
Agents first learn to decompose images into basic concepts via a discrete codebook in a multi-target coordination game (Decompose step, no distractors), then reuse that codebook to describe images with novel concept combinations (Compose step), with zero-shot composition observed in some cases.
Key results
- On Thing, CB-CtD reaches 1.00 accuracy, AMI, CI and CBM, with BOS 0.98.
- On Shape, CB-CtD reaches 0.99 accuracy with CBM 1.00 and CI 1.00; zero-shot CB-CtD-ZS gets 0.81 accuracy with CBM 1.00.
- On Mnist, zero-shot CB-CtD-ZS (0.89 accuracy, CBM 0.96) matches or surpasses CB-CtD (0.81 accuracy, CBM 0.95).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a Decompose phase before Compose (ablation) | CB communication · Thing | accuracy: 0.273 -> 1.000 | Helped |
| Add a Decompose phase before Compose (ablation) | CB communication · Thing | CBM: 0.258 -> 1.000 | Helped |
| Add a Decompose phase before Compose (ablation) | CB communication · Shape | accuracy: 0.213 -> 0.988 | Helped |
| Add a Decompose phase before Compose (ablation) | CB communication · Mnist | accuracy: 0.422 -> 0.811 | Helped |
| Add a Decompose phase before Compose (ablation) | CB communication · Qrc | accuracy: 0.98 -> 0.95 | Hurt |
| Add a Decompose phase before Compose (ablation) | CB communication · Qrc | CBM: 0.17 -> 0.17 | No clear effect |
| Skip Compose training after Decompose (ablation) | CB-CtD · Mnist | accuracy: 0.811 -> 0.89 | Helped |
| Use a single target instead of two or more during Decompose (ablation) | CB communication · Thing | compositionality | Hurt |
| Reduce the number of targets during Decompose (ablation) | GS and QT communication · Thing | accuracy | Helped |
| Vary beta_1 and beta_2 from 0.1 to 1.0 (ablation) | CtD · All game setups | results | No clear effect |
Surprising findings
- Zero-shot composition (CtD-ZS) is comparable to or better than additional Compose-step training on Mnist and Shape.
- On the non-compositional QRC dataset, zero-shot accuracy drops to 0.48 versus 0.95 with CtD.
Limitations
- Message length must match phrase length in the data; variable-length communication is not supported.
- Results on COCO are less conclusive with lower compositionality.
- Requires a predefined set of foundational concepts for the Decompose step.
Experimental setup
Datasets: Thing Shape MNIST COCO QRC
Benchmarks: Thing Shape MNIST COCO QRC
Metrics: accuracy AMI POS BOS CI CBM
Baselines: Gumbel-softmax (GS) Quantized communication (QT) C/D (compose without decompose) CB-CtD-ZS
Builds on: VQ-VAE Gumbel-softmax communication Diff multi-target game Concept Best Matching
Closest papers by meaning
- Learning Multi-Object Positional Relationships via Emergent Communication (AAAI 2024 · similarity 0.88)
- A compressive-expressive communication framework for compositional representations (NeurIPS 2025 · similarity 0.88)
- Iterated Learning Improves Compositionality in Large Vision-Language Models (CVPR 2024 · similarity 0.88)
- CraftFactory: A Conditioned Control Policy Benchmark for Compositional Generalization (AAAI 2025 · similarity 0.87)
- Scaling can lead to compositional generalization (NeurIPS 2025 · similarity 0.86)
- In-Context Compositional Learning vis Sparse Coding Transformer (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.