Research map / Large language models / Retrieval and knowledge
Enhancing Multi-Hop Fact Verification with Structured Knowledge-Augmented Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LLM-SKAN, which fine-tunes Llama2-7B as an entity-relation-entity triplet extractor and fuses the extracted triplets with claim/evidence nodes in a GAT-based relation graph for multi-hop and single-hop fact verification.
Problem
Multi-hop fact verification requires understanding complex relations between entities across multiple evidence pieces, which existing models (including LLMs used as direct predictors) handle poorly.
Approach
Use a LoRA-fine-tuned LLM only as a relation/triplet extractor (not as a verdict predictor), then build claim-evidence-entity relation graphs with edge features from triplets and fuse them with a small GAT for classification.
Key results
- LLM-SKAN reaches 79.90% accuracy on 2-hop HOVER versus 78.50 best prior (SaGP 77.90), and 79.25% on FEVER accuracy versus 78.47 for SaGP.
- On 4-hop HOVER, LLM-SKAN achieves 77.95% accuracy and 73.78 FEVER score, outperforming SaGP (76.01 / 72.66).
- Ablation shows removing the knowledge extractor drops 2-hop HOVER accuracy from 79.90 to 78.05.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the LLM-driven knowledge extractor and fine-grained entity-relation triplets (ablation) | LLM-SKAN · FEVER | accuracy: 78.08 -> 79.25 | Helped |
| Add the LLM-driven knowledge extractor and fine-grained entity-relation triplets (ablation) | LLM-SKAN · 2-hop HOVER | accuracy: 78.05 -> 79.90 | Helped |
| Add relation graph construction with relation edges (ablation) | LLM-SKAN · FEVER | accuracy: 77.04 -> 79.25 | Helped |
| Add relation graph construction with relation edges (ablation) | LLM-SKAN · 2-hop HOVER | accuracy: 77.17 -> 79.90 | Helped |
| Use relation graph fusion instead of sequential attention (ablation) | LLM-SKAN · FEVER | accuracy: 77.40 -> 79.25 | Helped |
| Use relation graph fusion instead of sequential attention (ablation) | LLM-SKAN · 2-hop HOVER | accuracy: 77.44 -> 79.90 | Helped |
| Use relation graph fusion instead of concatenation | LLM-SKAN · fact verification module analysis | performance | Helped |
| Fine-tune the LLM extractor rather than using prompts | LLM-SKAN knowledge extractor · fact verification module analysis | performance | Helped |
Surprising findings
- The paper states LLMs are poor at reasoning and are better used as extractors than predictors, and reports prompt-only LLM extraction leaves a large gap versus fine-tuned extraction.
Limitations
- Relies on a single Llama2-7B extractor and a small GNN; LLM reasoning is explicitly excluded from the pipeline.
- Multi-hop gains over prior graph methods are modest (roughly 1-2 accuracy points).
Experimental setup
Datasets: DocRED-FE FEVER HOVER
Benchmarks: FEVER 2-hop HOVER 3-hop HOVER 4-hop HOVER
Metrics: Accuracy FEVER score
Baselines: DeBERTa GEAR EvidenceNet CO-GAT SaGP HiSS MultiKE-GAT
Builds on: GEAR DocRED DeBERTa LoRA
Closest papers by meaning
- MRR-FV: Unlocking Complex Fact Verification with Multi-Hop Retrieval and Reasoning (AAAI 2025 · similarity 0.89)
- Beyond Text: Fine-Grained Multi-Modal Fact Verification with Hypergraph Transformers (AAAI 2025 · similarity 0.87)
- Unsupervised Pretraining for Fact Verification by Language Model Distillation (ICLR 2024 · similarity 0.87)
- Heterogeneous Graph Reasoning for Fact Checking over Texts and Tables (AAAI 2024 · similarity 0.87)
- Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers (ICML 2025 · similarity 0.86)
- Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.