Research map / Vision-language and multimodal / Multimodal LLMs
Calibrating Concepts and Operations: Towards Symbolic Reasoning on Real Images
Zhuowan Li, Elias Stengel-Eskin, Yixiao Zhang, Cihang Xie, Quan Hung Tran, Benjamin Van Durme, Alan Yuille
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add operation calibration (weighted merge) (ablation) | NSCL baseline · GQA balanced testdev (gold programs) | accuracy: 47.01 -> 51.30 | Improved |
| remove normalization of concept embeddings (keep magnitudes) (ablation) | NSCL + operation calibration · GQA balanced testdev (gold programs) | accuracy: 51.30 -> 54.65 | Improved |
| add concept calibration on top of unnormalized embeddings (ablation) | NSCL + operation calibration · GQA balanced testdev (gold programs) | accuracy: 54.65 -> 56.13 | Improved |
| add concept and operation calibration together (ablation) | NSCL · GQA balanced testdev (gold programs) | accuracy: 47.01 -> 56.13 | Improved |
| use symbolic scene representation instead of dense visual features (ablation) | CCO · GQA balanced testdev (gold programs) | accuracy: 53.00 -> 56.13 | Improved |
| replace MMN parser with MISO parser | CCO · GQA testdev_balanced | accuracy: 54.01 -> 54.11 | No clear effect |
| replace MMN parser with MISO parser | program parsing · GQA testdev_balanced | exact match: 85.13 -> 91.18 | Improved |
| add operation calibration (ablation) | CCO · GQA perturbed testdev | accuracy: 55.49 -> 56.13 | Improved |
| use gold scene graph instead of detected symbolic scene | CCO · GQA balanced testdev (gold programs) | accuracy: 56.13 -> 89.61 | Improved |
| remove low-weight operations from questions (all removable) | CCO · GQA perturbed testdev | accuracy: 56.13 -> -2.59 | Hurt |
| remove low-weight operations from questions (all removable) | CCO · GQA visual-hard split | accuracy: 37.42 -> -4.72 | Hurt |
Closest papers by meaning
- Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding (NeurIPS 2018 · similarity 0.89)
- Probabilistic Neural Symbolic Models for Interpretable Visual Question Answering (ICML 2019 · similarity 0.88)
- GENOME: Generative Neuro-Symbolic Visual Reasoning by Growing and Reusing Modules (ICLR 2024 · similarity 0.86)
- Concept-RuleNet: Grounded Multi-Agent Neurosymbolic Reasoning in Vision Language Models (AAAI 2026 · similarity 0.85)
- Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Models (ICLR 2025 · similarity 0.84)
- NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
