Research map / Large language models / Language, safety and interpretability
FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FactTest, a hypothesis-testing framework that uses Neyman-Pearson-style threshold calibration on entropy-based certainty scores to give finite-sample, distribution-free Type I error guarantees for LLM factuality, letting the model abstain on uncertain questions.
Problem
LLMs hallucinate with high confidence, and existing detection methods give no formal Type I error guarantees for deciding when a model's answer can be trusted in high-stakes settings.
Approach
Frames factuality as a hypothesis test and selects a score threshold from a labeled calibration set via order statistics so the false positive rate (answering wrongly while deeming it certain) is bounded at alpha with probability 1-delta; extends to covariate shift via density-ratio rejection sampling and proves Type II error bounds.
Key results
- With alpha=0.05, FactTest-kle-15 lifts OpenLLaMA-3B accuracy on ParaRel from 36.66% (pretrained) to 78.45%, and FactTest-ve-15 on OpenLLaMA-7B reaches 80.29%.
- Empirical Type I error stays below alpha=0.05 across most configurations (e.g., ParaRel OpenLLaMA-7B ve_15 at 0.0145), though some entries slightly exceed alpha (e.g., OpenLLaMA-3B ParaRel ve_5 at 0.0455 is below, but FEVER LLaMA-7B ve_5 at 0.0598 exceeds 0.05).
- FactTest-t (half training data) outperforms R-Tuning by 34% accuracy on HotpotQA and 28% on FEVER per the paper text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply FactTest-KLE-15 instead of answering all questions | OpenLLaMA-13B · ParaRel | accuracy: 42.21 -> 83.84 | Helped |
| Apply FactTest-SE-15 instead of answering all questions | LLaMA-13B · HotpotQA | accuracy: 30.83 -> 60.69 | Helped |
| Apply FactTest-SE-10 instead of answering all questions | OpenLLaMA-7B · WiCE | accuracy: 72.96 -> 100.0 | Helped |
| Apply FactTest-SE-15 instead of answering all questions | OpenLLaMA-3B · FEVER | accuracy: 39.74 -> 83.90 | Helped |
| Use OpenLLaMA-13B-derived KLE certainty scores with FactTest | GPT-4o mini · ParaRel | accuracy: 52.83 -> 85.84 | Helped |
| Use OpenLLaMA-13B-derived SE certainty scores with FactTest | GPT-4o mini · WiCE | accuracy: 75.67 -> 86.95 | Helped |
| Use FactTest-t with half the training data instead of R-Tuning with all training data | LLaMA-7B · ParaRel | accuracy: 89.65 -> 92.76 | Helped |
| Use FactTest-t with half the training data instead of R-Tuning | OpenLLaMA-3B · FEVER | accuracy: 67.19 -> 97.82 | Helped |
| Vary certainty score function and number of generated outputs (ablation) | FactTest · ParaRel, HotpotQA, WiCE, and FEVER | accuracy | Mixed |
| Increase significance level from α=0.05 to α=0.10 (ablation) | FactTest · ParaRel, HotpotQA, WiCE, and FEVER | accuracy | Mixed |
Surprising findings
- Some reported empirical Type I errors exceed the nominal alpha (e.g., 0.0598 for LLaMA-7B on FEVER at alpha=0.05; 0.1151 for OpenLLaMA-7B on WiCE at alpha=0.10), in tension with the stated guarantee that holds only with probability 1-delta.
Limitations
- Only three entropy-based certainty functions are implemented, which may not be optimal across tasks.
- Framework is offline; it constructs the certainty predictor in batch rather than online testing.
- Covariate shift extension assumes the density ratio is known or estimated and bounded by B.
Experimental setup
Datasets: ParaRel HotpotQA WiCE FEVER ParaRel-OOD
Benchmarks: ParaRel HotpotQA WiCE FEVER ParaRel-OOD
Metrics: accuracy on willingly answered questions Type I error (FPR) Type II error (FNR) answer rate
Baselines: Pretrained R-Tuning Finetune-All Finetune-Half
Builds on: R-Tuning Semantic Entropy Kernel Language Entropy Neyman-Pearson umbrella algorithm
Closest papers by meaning
- LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.92)
- Joint Evaluation of Answer and Reasoning Consistency for Hallucination Detection in Large Reasoning Models (AAAI 2026 · similarity 0.92)
- How Language Model Hallucinations Can Snowball (ICML 2024 · similarity 0.91)
- ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models (NeurIPS 2024 · similarity 0.91)
- LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations (ICLR 2025 · similarity 0.91)
- DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (ICLR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.