Research map / Large language models / Agents, code and math
Compositional-ARC: Assessing Systematic Generalization in Abstract Spatial Reasoning
Philipp Mondorf, Shijia Zhou, Monica Riedler, Barbara Plank
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add auxiliary copy task to MLC training (ablation) | MLC · Compositional-ARC Systematicity (mean of 4 splits) | exact match accuracy (%): 69.05 -> 86.73 | Improved |
| add primitive transformations to the study examples (ablation) | MLC · Compositional-ARC Systematicity (mean of 4 splits) | exact match accuracy (%): 75.27 -> 86.73 | Improved |
| add level-1 compositions to the study examples (ablation) | MLC · Compositional-ARC Systematicity (mean of 4 splits) | exact match accuracy (%): 21.01 -> 86.73 | Improved |
| add test-time training on study examples | Llama-3.2-3B-ReARC · Compositional-ARC Systematicity | exact match accuracy (%): 0.87 -> 73.70 | Improved |
| add test-time training on study examples | Mistral-NeMO-Minitron-8B-Full · Compositional-ARC Systematicity | exact match accuracy (%): 0.70 -> 78.20 | Improved |
| add image input to the text prompt | GPT-4o · Compositional-ARC 3-Shot | exact match accuracy (%): 22.28 -> 19.42 | Hurt |
| add image input to the text prompt | Gemini 2.0 Flash · Compositional-ARC 3-Shot | exact match accuracy (%): 30.08 -> 17.19 | Hurt |
| add image input to the text prompt | GPT-4o · Compositional-ARC Systematicity | exact match accuracy (%): 0.99 -> 0.86 | Hurt |
| add image input to the text prompt | Gemini 2.0 Flash · Compositional-ARC Systematicity | exact match accuracy (%): 2.66 -> 2.05 | Hurt |
| switch from 3-shot examples to systematic primitive+level-1 examples | MLC · Compositional-ARC | exact match accuracy (%): 99.92 -> 78.26 | Hurt |
| expand transformation variety (more translations, extensions, colors) | MLC · Compositional-ARC Systematicity | exact match accuracy (%): 88.10 | No clear effect |
| use MLC small model instead of o3-mini | Compositional-ARC Systematicity · Compositional-ARC Systematicity | exact match accuracy (%): 0.53 -> 78.26 | Improved |
Closest papers by meaning
- EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs (CVPR 2026 · similarity 0.86)
- Socratic-Geo: Synthetic Data Generation and Cross-Modal Geometric Reasoning via Multi-Agent Interaction (CVPR 2026 · similarity 0.85)
- Neural-Symbolic Recursive Machine for Systematic Generalization (ICLR 2024 · similarity 0.85)
- Learning to Compose: Improving Object Centric Learning by Injecting Compositionality (ICLR 2024 · similarity 0.85)
- Does Data Scaling Lead to Visual Compositional Generalization? (ICML 2025 · similarity 0.85)
- Provable Compositional Generalization for Object-Centric Learning (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
