Research map / Vision-language and multimodal / Visual question answering
Probabilistic Neural Symbolic Models for Interpretable Visual Question Answering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use probabilistic latent programs rather than the NMN baseline | VQA on SHAPES with 10% program supervision · SHAPES Test | VQA accuracy: 61.99±0.96 -> ~94.53±2.06 | Helped |
| Use probabilistic latent programs rather than the NMN baseline | VQA on CLEVR with 0.143% program supervision · CLEVR v1.0 test | VQA accuracy: 75.71 -> 97.73 | Helped |
| Use probabilistic latent programs rather than the NMN baseline | Program prediction on CLEVR with 0.143% program supervision · CLEVR validation, joint training | program prediction accuracy: 63.08±9.91 -> 93.87±8.73 | Helped |
| Add the KL-divergence regularization term (β>0) instead of setting β=0 (ablation) | Prob-NMN on SHAPES with 10% program supervision · SHAPES | program prediction accuracy: 23.14±6.1% -> 60.18±9.56% | Helped |
| Add the KL-divergence regularization term (β>0) instead of setting β=0 (ablation) | Prob-NMN on SHAPES with 10% program supervision · SHAPES validation, joint training | VQA accuracy: 74.82±17.88 -> 96.86±2.48 | Helped |
| Disable the KL-divergence term by setting β=0 (ablation) | Prob-NMN on SHAPES with 10% program supervision · SHAPES | question reconstruction accuracy: 83.60±5.57% -> 94.3±1.85% | Helped |
| Set β=1 and optimize the true ELBO rather than using β<1 (ablation) | Prob-NMN on SHAPES with 10% program supervision · SHAPES | question reconstruction accuracy: 83.60% -> 8.74% | Hurt |
| Set β=1 and optimize the true ELBO rather than using β<1 (ablation) | Prob-NMN on SHAPES with 10% program supervision · SHAPES | VQA accuracy: 74.45±15.58; drop of > 20 % | Hurt |
| Replace atomic predicate-specific modules with argument-based modules (ablation) | Prob-NMN on SHAPES with 15% supervision · SHAPES validation | VQA accuracy: 97.53 (± 1.23) | No clear effect |
| Use a syntactic program prior instead of an empirical prior (ablation) | Prob-NMN question coding on SHAPES with 5% supervision · SHAPES | question reconstruction accuracy: 38.39 ± 11.92% -> 54.86 ± 6.75% | Helped |
| Use a syntactic program prior instead of an empirical prior (ablation) | Prob-NMN on SHAPES with more program supervision · SHAPES | performance | Hurt |
| Average predictions over 20 program samples rather than use one sample (ablation) | Prob-NMN on out-of-distribution CLEVR-Humans · CLEVR-Humans validation | VQA accuracy: 50.26±2.01 (one sample); 50.81±1.93 (20 samples) | No clear effect |
Closest papers by meaning
- Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding (NeurIPS 2018 · similarity 0.92)
- Neuro-Symbolic Visual Reasoning: Disentangling "Visual" from "Reasoning" (ICML 2020 · similarity 0.91)
- Learning Answer Embeddings for Visual Question Answering (CVPR 2018 · similarity 0.89)
- Super-CLEVR: A Virtual Benchmark To Diagnose Domain Robustness in Visual Reasoning (CVPR 2023 · similarity 0.89)
- Consistency and Uncertainty: Identifying Unreliable Responses From Black-Box Vision-Language Models for Selective Visual Question Answering (CVPR 2024 · similarity 0.88)
- Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Models (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.