Research map / Vision-language and multimodal / Human and embodied modeling
A Benchmark for Compositional Visual Reasoning
Aimen Zerroug, Mohit Vaishnav, Julien Colin, Sebastian Musslick, Thomas Serre
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SSL pretraining, joint training | ResNet-50 · CVR 1000 samples | accuracy: 59.2 -> 79.2 | Improved |
| add SSL pretraining, individual training | ViT-small · CVR 1000 samples | accuracy: 35.1 -> 65.5 | Improved |
| add SSL pretraining, joint training | ViT-small · CVR 1000 samples | accuracy: 31.4 -> 60.7 | Improved |
| train jointly on all rules instead of individually | ResNet-50 · CVR 1000 samples | accuracy: 44.8 -> 59.2 | Improved |
| train jointly on all rules instead of individually | ViT-small · CVR 1000 samples | accuracy: 35.1 -> 31.4 | Hurt |
| add ResNet-18 backbone to SCL | SCL · CVR 1000 samples individual | accuracy: 33.4 -> 48.3 | Improved |
| joint training with SSL pretraining (SES) | ResNet-50 vs ViT-small · CVR | SES: 57.0 vs 44.7 | Improved |
| increase training data from 1000 to 10k with SSL joint | ResNet-50 · CVR | accuracy: 79.2 -> 93.1 | Improved |
| pretrain on elementary rules then finetune on compositions (curriculum) | baselines · CVR compositions | accuracy | Improved |
| test composition-trained models on elementary rules (reverse curriculum) | baselines · CVR elementary rules | accuracy | Hurt |
| compare models with humans at 20 samples | ResNet-50 SSL vs humans · CVR 45 tasks | accuracy: 45.7 vs 78.7 | Hurt |
| increase training samples from 20 to 1000 | ResNet-50 SSL · CVR 45 tasks | accuracy: 45.7 -> 78.3 | Improved |
Closest papers by meaning
- VisRes Bench: On Evaluating the Visual Reasoning Capabilities of VLMs (CVPR 2026 · similarity 0.90)
- Investigating Compositional Challenges in Vision-Language Models for Visual Grounding (CVPR 2024 · similarity 0.89)
- NePTune: A Neuro-Pythonic Framework for Tunable Compositional Reasoning on Vision-Language (ICLR 2026 · similarity 0.89)
- Learning to Compose: Improving Object Centric Learning by Injecting Compositionality (ICLR 2024 · similarity 0.89)
- ConMe: Rethinking Evaluation of Compositional Reasoning for Modern VLMs (NeurIPS 2024 · similarity 0.89)
- Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
