Research map / Vision-language and multimodal / Visual question answering
Graph Reasoning Transformers for Knowledge-Aware Question Answering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Graph Reasoning Transformers (GRT), which encode knowledge triplets as atomic units with a triplet-level graph encoder, align text and KG representations via pretraining, and fuse modalities with attention-biased cross-modal layers for knowledge-aware QA.
Problem
Knowledge-aware question answering with KGs fails to exploit relational semantics of triplets and suffers from a modality gap between text and KG representations that hinders cross-modal fusion.
Approach
Treat knowledge triplets (not nodes) as atomic units encoded by a triplet-level graph encoder; align text and triplet representations with text-triplet matching and MLM pretraining; fuse modalities via a cross-modal transformer whose attention is biased by entity-word links.
Key results
- GRT reaches 76.1% test accuracy on CommonsenseQA (IH split), above QAT (75.4%) and FIT (75.6%) and 4.1 points over fine-tuned RoBERTa-large (68.6%).
- GRT achieves 87.3% on OpenbookQA with facts, versus 86.9% for QAT, the previous best listed.
- GRT achieves 39.5% test accuracy on MedQA-USMLE, above GSC and QAT (39.3%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use triplet-level graph features instead of node features (ablation) | GRT · CSQA | accuracy: 73.1 -> 76.1 | Helped |
| use triplet-level graph features instead of node features with a GNN (ablation) | GRT · CSQA | accuracy: 74.1 -> 76.1 | Helped |
| use triplet-level graph features instead of node features (ablation) | GRT · OBQA | accuracy: 68.4 -> 72.6 | Helped |
| use triplet-level graph features instead of node features with a GNN (ablation) | GRT · OBQA | accuracy: 68.8 -> 72.6 | Helped |
| use triplet-level graph features instead of node features (ablation) | GRT · OBQA w/ facts | accuracy: 84.2 -> 87.3 | Helped |
| use triplet-level graph features instead of node features with a GNN (ablation) | GRT · OBQA w/ facts | accuracy: 85.4 -> 87.3 | Helped |
| use triplet-level graph features instead of node features (ablation) | GRT · Negations | accuracy: 74.3 -> 79.7 | Helped |
| use triplet-level graph features instead of node features with a GNN (ablation) | GRT · Negations | accuracy: 75.6 -> 79.7 | Helped |
| add representation alignment pretraining (ablation) | GRT · CommonsenseQA and OpenbookQA | accuracy | Helped |
| use a cross-attention layer without explicit supervision instead of no information fusion (ablation) | GRT · OpenbookQA | accuracy | Hurt |
| add cross-modal information fusion with attention bias (ablation) | GRT · OpenbookQA | accuracy | Helped |
| use triplet-level graph features instead of node features (ablation) | GRT · Entities > 7 | accuracy: 79.6 -> 80.0 | Helped |
Surprising findings
- A cross-attention layer without explicit alignment supervision underperforms the no-fusion variant on OpenbookQA, which the authors attribute to limited training corpora.
Limitations
- Gains on MedQA-USMLE are small (39.5% vs 39.3% for GSC and QAT).
- Relies on entity linking to produce KG triplets and text-triplet correspondences, so pretraining depends on linking quality.
- Results reported are single-run means with standard deviations on relatively small benchmarks.
Experimental setup
Datasets: CommonsenseQA OpenbookQA MedQA-USMLE ConceptNet
Benchmarks: CommonsenseQA OpenbookQA MedQA-USMLE
Metrics: accuracy
Baselines: RoBERTa-large (w/o KG) GconAttn RN MHGRN QA-GNN SAFE GreaseLM GSC QAT FIT ClinicalBERT BioBERT
Builds on: QA-GNN GreaseLM RoBERTa TransE SapBERT
Closest papers by meaning
- Debate on Graph: A Flexible and Reliable Reasoning Framework for Large Language Models (AAAI 2025 · similarity 0.90)
- Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation (AAAI 2025 · similarity 0.90)
- Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models (ICML 2025 · similarity 0.89)
- EPERM: An Evidence Path Enhanced Reasoning Model for Knowledge Graph Question and Answering (AAAI 2025 · similarity 0.88)
- Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.88)
- Knowledge Graph Prompting for Multi-Document Question Answering (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.