Research map / Vision-language and multimodal / Visual question answering
Object Attribute Matters in Visual Question Answering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OAM-VQA, which fuses object attributes with visual features via a multimodal graph neural network (attribute fusion module) and distills implicit knowledge from VLP models into attributes with a contrastive loss, on top of LXMERT for visual question answering.
Problem
Standard VQA models integrate visual and textual semantics mainly through attention, lacking object-level visual-language alignment, which hurts fine-grained counting questions and scene understanding and OOD robustness.
Approach
Uses object attributes as the bridge between modalities: a two-subgraph multimodal GNN passes messages between visual object nodes and attribute nodes, and a contrastive loss distills implicit knowledge from OFA, BLIP and BLIP2 prompts into attribute features.
Key results
- Achieves 75.22% overall accuracy on COCO-QA, outperforming MRA-Net (70.27%), with number-question accuracy rising to 68.20%.
- Reaches 60.97% on VQA-CP v2 test, a +9.12 point gain over LXMERT (51.85%).
- Reaches 65.43% on VQA-CP v1 and 90.62% on TDIUC, the highest among reported methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OAM-VQA instead of MRA-Net | COCO-QA · COCO-QA | accuracy (All): 70.27 -> 75.22 | Helped |
| use OAM-VQA instead of MuRel | TDIUC · TDIUC | accuracy: surpassing MuRel 2.42% | Helped |
| use OAM-VQA instead of CL | VQA-CPv1 · VQA-CPv1 | accuracy: outperforms the CL method by 4.16% | Helped |
| use OAM-VQA instead of the strongest listed baseline, MDDC | VQA-CPv2 · VQA-CPv2 test | accuracy (All): 60.97 vs 69.77 | Hurt |
| use OAM-VQA instead of the strongest listed baseline, MDDC | VQAv2 · VQAv2 val | accuracy (All): 71.99 vs 74.51 | Hurt |
| use OAM-VQA instead of LXMERT† | VQAvs · VQAvs | Average: 53.16 -> 53.71 | Helped |
| add CKDM to LXMERT (ablation) | LXMERT · COCO-QA | accuracy: 72.61 -> 74.94 | Helped |
| add AFM to LXMERT (ablation) | LXMERT · COCO-QA | accuracy: +2.58% on COCO-QA | Helped |
| add CKDM to LXMERT (ablation) | LXMERT · VQA-CPv1 | accuracy: 52.21 -> 63.97 | Helped |
| add AFM to LXMERT (ablation) | LXMERT · VQA-CPv1 | accuracy: +12.88% on VQA-CPv1 | Helped |
| add CKDM to LXMERT (ablation) | LXMERT · VQA-CPv2 | accuracy: 51.85 -> 58.33 | Helped |
| add AFM to LXMERT (ablation) | LXMERT · VQA-CPv2 | accuracy: +8.34% on VQA-CPv2 | Helped |
Limitations
- Relies on multiple external VLP models and an MLLM (BLIP2, mPLUG-Owl, OFA, BLIP) to generate knowledge and descriptions, adding pipeline complexity.
- Compares mainly against LXMERT-based and debiasing baselines; VQA-CPv2 overall accuracy is below some LXMERT-based SOTA methods such as MDDC (69.77) and MUTANT (69.52).
Experimental setup
Datasets: COCO-QA VQAv2 VQA-CPv2 VQA-CPv1 VQAvs TDIUC
Benchmarks: COCO-QA TDIUC VQA-CPv1 VQA-CPv2 VQAv2 VQAvs
Metrics: accuracy
Baselines: LXMERT MRA-Net MUTANT MDDC MuRel SAR Re-scaling CSS LMH UpDn
Builds on: LXMERT TwO compound token attention LRC-BERT MMBS
Closest papers by meaning
- Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Models (ICLR 2025 · similarity 0.90)
- VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning (CVPR 2024 · similarity 0.90)
- Core-to-Global Reasoning for Compositional Visual Question Answering (AAAI 2025 · similarity 0.90)
- Bridging the Gap between 2D and 3D Visual Question Answering: A Fusion Approach for 3D VQA (AAAI 2024 · similarity 0.89)
- When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.89)
- Multimodal Commonsense Knowledge Distillation for Visual Question Answering (Student Abstract) (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.