Research map / Large language models / Retrieval and knowledge
OntoFact: Unveiling Fantastic Fact-Skeleton of LLMs via Ontology-Driven Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OntoFact, an ontology-driven framework that uses actor-critic and discrete RL over KG ontology and instance triples to generate error-prone yes/no test cases and detect LLM factual gaps, releasing five fact-detection benchmarks.
Problem
Detecting missing or incorrect factual knowledge (intrinsic hallucination) in LLMs across many domains, given that prior QA-based factuality tests cover only a few domains and suffer from yes-bias and overconfidence.
Approach
Use ontology-level KG triples as probes: a two-view RL mechanism (discrete instance-view policy gradient plus continuous ontology-view DDPG-style actor-critic) navigates the KG to find un-factual ontology regions, while a hallucination-free detection strategy adds simple and hard negative samples to counter yes-bias.
Key results
- ChatGPT shows error proportions of 51.6% on DBpedia and 64.7% on YAGO (average EP 50.1% across datasets as stated).
- ORL achieves F1 scores of roughly 70%-90% for most LLMs and an average recall of about 80%.
- ORL reduces evaluation time by 35.29% to 63.12% relative to exhaustive testing.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ontology-driven reinforcement learning to select error-prone test cases instead of exhaustive testing | OntoFact · DBpedia | evaluation time: 267.04 -> 109.53 | Helped |
| Use ontology-driven reinforcement learning to select error-prone test cases instead of exhaustive testing | OntoFact · YAGO | evaluation time: 330.75 -> 209.96 | Helped |
| Use ORL to predict un-factual ontology triples | OntoFact · DBpedia | F1 (%): 99.9 | Helped |
| Use ORL to predict un-factual ontology triples | OntoFact · YAGO | F1 (%): 56.9%-100% | Helped |
| Evaluate triples with HFD by including two negative samples rather than modifying a single entity once | OntoFact · 5 benchmarks | EP improvement: 13% | Helped |
| Use larger model parameter scales | LLMs · English general datasets | EP (%): 2.1%; 3.9%; 14.6% | Mixed |
Surprising findings
- Dominant LLMs are biased toward answering Yes regardless of whether the fact is in the knowledge base.
- ChatGPT is more factual on Chinese benchmarks (average EP 29.4% lower) than on English general benchmarks.
Limitations
- Relies on an NLI model for auxiliary judgment, with 97% agreement with human judgment on 500 samples.
- Test case templates are generated by ChatGPT and KG-based ground truth may itself contain errors (e.g., ~5% in YAGO).
Experimental setup
Datasets: DBpedia YAGO 4.5 CN-DBpedia BIOS 2.2 (ENG) BIOS 2.2 (CHS)
Benchmarks: DBpedia YAGO 4.5 CN-DBpedia BIOS 2.2 (ENG) BIOS 2.2 (CHS)
Metrics: error proportion (EP) accuracy precision recall F1 graph search time reduction
Baselines: Random exhaustive testing (w/o ORL)
Builds on: DDPG JOIE Policy gradient (REINFORCE) Qin et al. 2023 discriminative QA factuality
Closest papers by meaning
- FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees (ICML 2025 · similarity 0.89)
- Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-Based Retrofitting (AAAI 2024 · similarity 0.88)
- Long-form factuality in large language models (NeurIPS 2024 · similarity 0.88)
- LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations (ICLR 2025 · similarity 0.87)
- ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation (NeurIPS 2025 · similarity 0.87)
- Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models (AAAI 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.