Research map / Large language models / Agents, code and math
A compressive-expressive communication framework for compositional representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CELEBI, a self-supervised emergent-communication framework with progressive decoding, final-state imitation, and pairwise distance maximization that yields more compositional and efficient discrete messages on Shapes3D and MPI3D.
Problem
Neural agents struggle to learn compositional representations; emergent communication with iterated learning needs to balance expressivity and compressibility to produce compositional discrete messages for image reconstruction.
Approach
Extends the Lewis reconstruction game with iterated learning by (1) a prefix-wise reconstruction loss (progressive decoding), (2) imitation that matches only the teacher's final reconstruction rather than messages (final-state imitation), and (3) a batch pairwise cosine-dissimilarity regularizer over message symbol distributions (pairwise distance maximization) linked to entropy maximization.
Key results
- Progressive Decoding reduces useful length on Shapes3D from 10.0 to 7.5 (25% reduction) with TopSim rising from 0.244 to 0.270.
- PD+FiSI+PDM (lambda=1.5) achieves the best Shapes3D TopSim of 0.292 and useful length 7.0 versus 10.0 for the baseline.
- On MPI3D, VAE+CELEBI obtains DCI 0.137 vs 0.114 for VAE+EL and RMSE 14.82 vs 15.97, while beta-VAE/beta-TCVAE get higher categorical accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add progressive decoding (ablation) | Full message reconstruction baseline · Shapes3D | Useful length: 10.0 ± 0.0 -> 7.5 ± 0.215 | Helped |
| Add progressive decoding (ablation) | Full message reconstruction baseline · MPI3D | Last symbol MSE: 0.015 ± 0.0 -> 0.015 ± 0.0 | No clear effect |
| Add progressive decoding (ablation) | Full message reconstruction baseline · Shapes3D | Last symbol MSE: 0.212 ± 0.003 -> 0.238 ± 0.007 | Hurt |
| Use final-state imitation instead of message imitation (ablation) | Iterated learning communication · MPI3D | TopSim: 0.135 ± 0.001 -> 0.156 ± 0.001 | Helped |
| Use final-state imitation instead of message imitation (ablation) | Iterated learning communication · Shapes3D | TopSim: 0.257 ± 0.005 -> 0.283 ± 0.003 | Helped |
| Use final-state imitation instead of message imitation (ablation) | Iterated learning communication · MPI3D | Last symbol MSE: 0.016 ± 0.0 -> 0.02 ± 0.0 | Hurt |
| Imitate all receiver states instead of only the final state (ablation) | Iterated learning communication · Shapes3D | TopSim: 0.283 ± 0.003 -> 0.256 ± 0.003 | Hurt |
| Replace KoLeo with PDM (ablation) | PD+FiSI regularization · Shapes3D | TopSim: 0.256 ± 0.002 -> 0.292 ± 0.002 | Helped |
| Replace KoLeo with PDM (ablation) | PD+FiSI regularization · MPI3D | Useful length: 8.9 ± 0.221 -> 9.0 ± 0.167 | Hurt |
| Add PDM to PD+FiSI (ablation) | PD+FiSI · Shapes3D | TopSim: 0.283 ± 0.003 -> 0.292 ± 0.002 | Helped |
| Add PDM to PD+FiSI (ablation) | PD+FiSI · MPI3D | Useful length: 6.6 ± 0.341 -> 9.0 ± 0.167 | Hurt |
| Use λ=1.5 for progressive decoding (ablation) | Progressive decoding · Shapes3D and MPI3D | Useful length | Mixed |
Surprising findings
- Final-state imitation reaches higher TopSim than message imitation with minimal reconstruction loss, suggesting message-level matching constrains compositional structure too strongly.
- On MPI3D, the PDM regularizer does not achieve the highest TopSim and increases useful length relative to PD alone.
Limitations
- Experiments restricted to synthetic datasets Shapes3D and MPI3D with clean disentangled factors and no observational noise.
- MSE used as reconstruction loss may be inappropriate for natural images.
- Visual backbone (VAE) frozen across EL variants, which may bias comparison with jointly trained continuous baselines.
Experimental setup
Datasets: Shapes3D MPI3D
Benchmarks: Shapes3D MPI3D
Metrics: Topographic Similarity useful length reconstruction MSE DCI disentanglement RMSE classification accuracy
Baselines: Full message reconstruction (no IL) Message imitation Full state imitation PD+FiSI+KoLeo beta-VAE beta-TCVAE VAE + EL
Builds on: Lewis signaling game iterated learning EGG framework Gumbel-Softmax KoLeo entropy estimator
Closest papers by meaning
- Compositional languages emerge in a neural iterated learning model (ICLR 2020 · similarity 0.91)
- Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input (ICLR 2018 · similarity 0.89)
- CtD: Composition through Decomposition in Emergent Communication (ICLR 2025 · similarity 0.88)
- Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments (ICLR 2024 · similarity 0.87)
- Iterated Learning Improves Compositionality in Large Vision-Language Models (CVPR 2024 · similarity 0.87)
- Learning Multi-Object Positional Relationships via Emergent Communication (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.