Research map / Vision-language and multimodal / Visual question answering
Core-to-Global Reasoning for Compositional Visual Question Answering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CTGR, a core-to-global reasoning model for compositional VQA that filters redundant global image/question features using core (object/attribute/relation and predicate) features and fuses them hierarchically via bilinear attention.
Problem
Compositional VQA requires multimodal semantic understanding and logical reasoning, but enriched visual and question features contain redundancy that hurts cross-modal alignment and answer accuracy.
Approach
Extracts both global and core semantic features (objects, attributes, relations from an improved scene graph generator and question predicates via a stop-word filter), uses core text features and visual core features to filter redundancy in global features, then fuses core-level and global-level representations with hierarchical bilinear attention for answer prediction.
Key results
- CTGR reaches 75.0% on GQA validation and 72.9% on test, versus 72.1% for CFR on test.
- On GQA-sub, CTGR reaches 72.75% ACC(sub) versus 70.31% for DIS, with RC(1)-RC(3) gains of 0.60, 1.23, 1.91 points over DIS.
- CTGR achieves 73.3% on VQA2.0 test and 72.3% on Visual7W test, the best reported among compared models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CFR with CTGR | Compositional VQA models · GQA | accuracy: 73.6 -> 75.0 (validation); 72.1 -> 72.9 (test) | Helped |
| replace DIS with CTGR | GQA-sub · GQA-sub validation | ACC(sub): 70.31 -> 72.75 | Helped |
| replace DIS with CTGR | GQA-sub · GQA-sub validation | RC(3): 48.86 -> 50.77 | Helped |
| replace CFR with CTGR | VQA2.0 · VQA2.0 | accuracy: 69.7 -> 71.0 (validation); 72.5 -> 73.3 (test) | Helped |
| replace CFR with CTGR | Visual7W · Visual7W | accuracy: 69.8 -> 70.5 (validation); 71.9 -> 72.3 (test) | Helped |
| use CTGR core features instead of Coarse2Fine features (ablation) | CTGR feature embedding · GQA validation | accuracy: 66.7 -> 68.0 | Helped |
| add question feature filtering (ablation) | CTGR · GQA validation | accuracy: 68.0 -> 68.9 | Helped |
| add image feature filtering (ablation) | CTGR · GQA validation | accuracy: 68.0 -> 70.2 | Helped |
| add both text and image feature filtering (ablation) | CTGR · GQA validation | accuracy: 68.0 -> 70.8 | Helped |
| use hierarchical bilinear attention instead of bilinear attention (ablation) | CTGR multimodal fusion · GQA validation | accuracy: 72.7 -> 75.0 | Helped |
| set λ to 6 for selecting visual core features (ablation) | CTGR · GQA validation | accuracy: 70.5 (λ=2) -> 75.0 (λ=6); 75.0 (λ=6) -> 74.0 (λ=9) | Helped |
Limitations
- Gains over the strongest prior baseline (CFR) are modest, around 0.4-1.4 points.
- Relies on an SGG framework and stop-word filter for core feature extraction, whose noise is not analyzed in depth.
- Top-K hyperparameter lambda must be tuned; larger values add trainable parameters with little gain.
Experimental setup
Datasets: GQA GQA-sub VQA2.0 Visual7W
Benchmarks: GQA GQA-sub VQA2.0 Visual7W
Metrics: accuracy reasoning consistency RC(K)
Baselines: BUTD BAN CTI RCVQA MCAN MMN LXMERT OSCAR CFR MAC LCGN DIS
Builds on: CFR BUTD BAN Tang et al. scene graph generation framework
Closest papers by meaning
- Maintaining Reasoning Consistency in Compositional Visual Question Answering (CVPR 2022 · similarity 0.93)
- Object Attribute Matters in Visual Question Answering (AAAI 2024 · similarity 0.90)
- When Open-Vocabulary Visual Question Answering Meets Causal Adapter: Benchmark and Approach (AAAI 2025 · similarity 0.90)
- Separation of Powers: On Segregating Knowledge from Observation in LLM-enabled Knowledge-based Visual Question Answering (CVPR 2025 · similarity 0.89)
- VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning (CVPR 2024 · similarity 0.89)
- Towards Robust Visual Question Answering via Prompt-Driven Geometric Harmonization (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.