Research map / Vision-language and multimodal / Multimodal LLMs
Inferring and Executing Programs for Visual Reasoning
Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Judy Hoffman, Li Fei-Fei, C. Lawrence Zitnick, Ross Girshick
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use all 700K ground-truth programs instead of 18K (ablation) | program generator + execution engine · CLEVR (overall) | accuracy: 95.4 -> 96.9 | Improved |
| use 18K ground-truth programs instead of 9K (ablation) | semi-supervised model · CLEVR (overall) | accuracy: 88.6 -> 95.4 | Improved |
| replace the linear transform with an MLP (ablation) | CNN+LSTM+SA · CLEVR (overall) | accuracy: 69.8 -> 73.2 | Improved |
| train the execution engine on predicted rather than ground-truth programs (ablation) | execution engine · CLEVR (overall) | accuracy: loss of about 3 points | Hurt |
| finetune jointly on short and long questions without programs (ablation) | Ours (25K prog.) · CLEVR long questions | accuracy: 55.3 -> 77.8 | Improved |
| finetune jointly on short and long questions (ablation) | CNN+LSTM+SA+MLP · CLEVR long questions | accuracy: 64.3 -> 67.8 | Improved |
| train on short questions only versus baseline (ablation) | Ours (25K prog.) · CLEVR long questions | accuracy: 55.3 vs 64.3 for CNN+LSTM+SA+MLP | Hurt |
| finetune the program generator on CLEVR-Humans (ablation) | Ours (18K prog.) · CLEVR-Humans test | accuracy: 54.0 -> 66.6 | Improved |
| finetune on CLEVR-Humans (ablation) | CNN+LSTM+SA+MLP · CLEVR-Humans test | accuracy: 50.4 -> 57.6 | Improved |
| remove Batch Normalization from modules (ablation) | execution engine modules · CLEVR (overall) | accuracy | Improved |
Closest papers by meaning
- CodeDance: A Dynamic Tool-integrated MLLM for Executable Visual Reasoning (CVPR 2026 · similarity 0.90)
- ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both (NeurIPS 2026 · similarity 0.88)
- GENOME: Generative Neuro-Symbolic Visual Reasoning by Growing and Reusing Modules (ICLR 2024 · similarity 0.88)
- CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization (CVPR 2026 · similarity 0.88)
- Learning to Reason Iteratively and Parallelly for Complex Visual Reasoning Scenarios (NeurIPS 2024 · similarity 0.88)
- SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
