Research map / Large language models / Retrieval and knowledge

Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modeling

AAAI 2025 · Large language models › Retrieval and knowledge · approach: Knowledge editing in LLMs · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GCA, a zero-resource black-box hallucination detector for long-form LLM text that extracts knowledge triples, models contextual triple dependencies with an RGCN over sampled-response graphs, and adds LLM-based reverse verification via three triple-reconstruction tasks.

Problem

Detecting hallucinations in long open-ended generations without external resources is hard, since facts are hard to align across sampled responses and dependencies between facts are ignored, and divide-and-conquer methods suffer from omission issues.

Approach

Segment responses into knowledge triples for alignment, build a graph per response and apply RGCN message passing so each triple's consistency score reflects its contextual triples, and add reverse verification by reconstructing each triple's head entity, relation, and tail entity with constrained LLM prompts.

Key results

• GCA achieves F1 55.4 / Acc 68.3 on PHD, beating the best baseline RVQG (52.3 / 65.3).
• On WikiBio GCA reaches F1 90.7 / Acc 83.2 versus RVQG 85.7 / 79.2; on sub-WikiBio F1 90.5 / Acc 85.0.
• Removing RVF drops PHD F1 from 55.4 to 38.1, showing reverse verification matters most on PHD, though it does not help on WikiBio.

Measured effects reported in the paper

Surprising findings

• Removing the reverse verification module (RVF) leaves WikiBio performance unchanged, which the authors attribute to the dataset's 95% hallucination rate.
• The NLI-based SelfCheckGPT variant performs worst, below its embedding-based BERTScore variant.

Limitations

• WikiBio has heavy class imbalance (about 95% hallucinations) and pseudo-labels at passage level, which weakens some ablation conclusions.
• Relies on GPT-4 for triple extraction and reverse verification, which adds cost and dependence on a closed model.
• Baseline thresholds are tuned per dataset, and the evaluation uses small datasets (300, 238 and 12/48 samples).

Experimental setup

Datasets: PHD WikiBio sub-WikiBio

Benchmarks: PHD WikiBio sub-WikiBio TripleCom

Metrics: F1 Accuracy

Baselines: RVQG Semantic Entropy SelfCheckGPT-BERTScore SelfCheckGPT-NLI Self-contradiction Focus

Builds on: SelfCheckGPT reverse validation RGCN Sentence-BERT

Closest papers by meaning

• Enhancing Uncertainty Modeling with Semantic Graph for Hallucination Detection (AAAI 2025 · similarity 0.92)
• ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability (ICLR 2025 · similarity 0.91)
• Scalable Token-Level Hallucination Detection in Large Language Models (NeurIPS 2026 · similarity 0.91)
• LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.91)
• Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models (AAAI 2026 · similarity 0.91)
• G2LDetect: A Global-to-Local Approach for Hallucination Detection (AAAI 2025 · similarity 0.90)