Research map / Vision-language and multimodal / Human and embodied modeling
A Benchmark for Systematic Generalization in Grounded Language Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add GECA compositional data augmentation | multimodal seq2seq baseline · gSCAN C: Red squares | exact match (%): 23.51 -> 78.77 | Helped |
| add GECA compositional data augmentation | multimodal seq2seq baseline · gSCAN B: Yellow squares | exact match (%): 54.96 -> 34.92 | Hurt |
| add GECA compositional data augmentation | multimodal seq2seq baseline · gSCAN A: Random split | exact match (%): 97.69 -> 87.6 | Hurt |
| add GECA compositional data augmentation | multimodal seq2seq baseline · gSCAN H: Adverb to verb | exact match (%): 22.70 -> 11.83 | Hurt |
| add GECA compositional data augmentation | multimodal seq2seq baseline · gSCAN E: Relativity | exact match (%): 35.02 -> 33.19 | No clear effect |
| add GECA compositional data augmentation | multimodal seq2seq baseline · gSCAN D: Novel direction | exact match (%): 0.00 -> 0.00 | No clear effect |
| add GECA compositional data augmentation | multimodal seq2seq baseline · gSCAN F: Class inference | exact match (%): 92.52 -> 85.99 | No clear effect |
| increase number of 'cautiously' training examples k from 1 to 50 (ablation) | multimodal seq2seq baseline · gSCAN G: Adverb few-shot | exact match (%): 0.00 -> 4.63 | Helped |
| GECA when size adjective included in referring expression (ablation) | GECA on red squares split · gSCAN C: Red squares | exact match (%): 97.95 -> 78.64 | Hurt |
Closest papers by meaning
- CraftFactory: A Conditioned Control Policy Benchmark for Compositional Generalization (AAAI 2025 · similarity 0.89)
- Neural-Symbolic Recursive Machine for Systematic Generalization (ICLR 2024 · similarity 0.86)
- Attention as a Hypernetwork (ICLR 2025 · similarity 0.86)
- Quantifying Generalization Complexity for Large Language Models (ICLR 2025 · similarity 0.86)
- Potemkin Understanding in Large Language Models (ICML 2025 · similarity 0.85)
- Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning (ICLR 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.