Research map / Vision-language and multimodal / Visual question answering
Multi-Modal Graph Neural Network for Joint Reasoning on Vision and Scene Text
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a three-subgraph multi-modal GNN with Visual-Semantic, Semantic-Semantic, and Semantic-Numeric aggregation (ablation) | TextVQA model · TextVQA val OCR questions | accuracy (%): 35.38 -> 43.36 | Helped |
| Use a three-subgraph multi-modal GNN with Visual-Semantic, Semantic-Semantic, and Semantic-Numeric aggregation (ablation) | TextVQA model · TextVQA val Overall | accuracy (%): 27.55 -> 31.21 | Helped |
| Add Visual-Semantic aggregation (ablation) | MM-GNN · TextVQA val OCR questions | accuracy (%): 41.99 -> 43.36 | Helped |
| Add Semantic-Semantic aggregation (ablation) | MM-GNN · TextVQA val Overall | accuracy (%): 30.44 -> 31.21 | Helped |
| Add Semantic-Numeric aggregation (ablation) | MM-GNN · TextVQA val Overall | accuracy (%): 30.99 -> 31.21 | Helped |
| Use a single graph with vanilla aggregation instead of modality-specific subgraphs and aggregators (ablation) | TextVQA model · TextVQA val Overall | accuracy (%): 28.58 vs 31.21 | Hurt |
| Replace feature concatenation with element-wise summation (ablation) | MM-GNN · TextVQA val OCR questions | accuracy (%): 43.36 -> 40.40 | Hurt |
| Replace feature concatenation with element-wise product (ablation) | MM-GNN · TextVQA val OCR questions | accuracy (%): 43.36 -> 32.18 | Hurt |
| Replace feature concatenation with concatenation followed by an MLP (ablation) | MM-GNN · TextVQA val OCR questions | accuracy (%): 43.36 -> 38.44 | Hurt |
| Change the order of the three aggregators (ablation) | MM-GNN · TextVQA val Overall | accuracy (%): 30.27 to 30.65; 31.21 | No clear effect |
| Add MM-GNN to the TextVQA model (ablation) | TextVQA model · TextVQA val others-type questions | accuracy (%): 22.65 -> 27.36 | Helped |
Closest papers by meaning
- Object Attribute Matters in Visual Question Answering (AAAI 2024 · similarity 0.88)
- Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models (ICML 2025 · similarity 0.87)
- Multimodal Commonsense Knowledge Distillation for Visual Question Answering (Student Abstract) (AAAI 2025 · similarity 0.87)
- VTQA: Visual Text Question Answering via Entity Alignment and Cross-Media Reasoning (CVPR 2024 · similarity 0.87)
- Cross-modal Information Flow in Multimodal Large Language Models (CVPR 2025 · similarity 0.87)
- Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.