Research map / Large language models / Language, safety and interpretability
HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HaloScope, which estimates membership of unlabeled LLM generations via top singular vectors of centered embeddings and trains a truthfulness classifier on the resulting pseudo-labels for hallucination detection.
Problem
Hallucination detection for LLMs is hard because labeled truthful/hallucinated generations are scarce and costly to annotate, while unlabeled in-the-wild generations are a mixture of truthful and hallucinated content with no clear membership.
Approach
Identify a hallucination subspace from the top singular vectors of centered LLM embeddings of unlabeled generations, score each sample by its singular-value-weighted projection norm to estimate membership, then train an MLP truthfulness classifier on the split sets.
Key results
- On LLaMA-2-7b-chat, HaloScope reaches 78.64% AUROC on TruthfulQA vs. 68.00-ish for the best prior baseline CCS* (67.95) and 62.17 for Semantic Entropy, approaching the supervised oracle at 81.04%.
- On OPT-6.7b, HaloScope achieves 73.17% AUROC on TruthfulQA, compared with 63.91 for CCS*.
- On LLaMA-2-chat-13b, HaloScope achieves 80.37% AUROC on TruthfulQA and 95.68% on TydiQA-GP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HaloScope membership-subspace estimation and classifier instead of Semantic Entropy | LLaMA-2-7B-chat hallucination detection · TruthfulQA | AUROC: 62.17 -> 78.64 | Helped |
| Use HaloScope membership-subspace estimation and classifier instead of CCS* | LLaMA-2-7B-chat hallucination detection · TydiQA-GP | AUROC: 80.38 -> 94.04 | Helped |
| Remove singular-value weighting from membership score (ablation) | HaloScope · TruthfulQA | AUROC: 77.24 -> 78.64 | Hurt |
| Sum scores across layers instead of selecting the best layer (ablation) | HaloScope · TruthfulQA | AUROC: 65.82 -> 78.64 | Hurt |
| Use Attn(f) embeddings instead of transformer-block output embeddings (ablation) | HaloScope on LLaMA-2-7B-chat · TruthfulQA | AUROC: 75.63 -> 78.64 | Hurt |
| Use Q Attn(f) embeddings instead of transformer-block output embeddings (ablation) | HaloScope on LLaMA-2-7B-chat · TruthfulQA | AUROC: 76.06 -> 78.64 | Hurt |
| Use multinomial sampling at temperature 0.5 rather than greedy sampling for unlabeled generations (ablation) | HaloScope on LLaMA-2-7B-chat · TruthfulQA | AUROC: 76.62 -> 78.64 | Hurt |
| Reduce unlabeled generation count from 512 to 100 (ablation) | HaloScope on LLaMA-2-7B-chat · TruthfulQA | AUROC: 73.34 vs. 78.64 | Hurt |
| Extract the membership subspace from TriviaQA rather than TruthfulQA (ablation) | HaloScope · TruthfulQA | AUROC: 76.26 vs. 78.64 | Hurt |
| Scale the base model from OPT-6.7B to OPT-13B | HaloScope · TruthfulQA | AUROC: 73.17 -> 82.41 | Helped |
Surprising findings
- Summing layer-wise membership scores performs significantly worse than using a single best layer, attributed to low separability in top and bottom layers.
- Prompting LLMs to self-assess truthfulness (Verbalize, Self-evaluation) performs near chance on many datasets, attributed to overconfidence.
Limitations
- Relies on the assumption that hallucinated samples form a small subspace with larger projection norms, and the membership threshold T and layer selection require a labeled validation set (100 QA pairs).
- Performance is sensitive to the number of unlabeled samples, degrading with fewer generations (e.g., 73.34 at N=100 vs 78.64 at N=512).
- Evaluated only on QA-style tasks with four datasets and two model families up to 13B parameters.
Experimental setup
Datasets: TruthfulQA TriviaQA CoQA TydiQA-GP WikiBio HaluEval
Benchmarks: TruthfulQA TriviaQA CoQA TydiQA-GP
Metrics: AUROC
Baselines: Perplexity LN-Entropy Semantic Entropy Lexical Similarity EigenScore SelfCKGPT Verbalize Self-evaluation CCS CCS*
Builds on: Contrast-Consistent Search Huber contamination model EigenScore
Closest papers by meaning
- Steer LLM Latents for Hallucination Detection (ICML 2025 · similarity 0.92)
- LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.92)
- Robust Hallucination Detection in LLMs via Adaptive Token Selection (NeurIPS 2025 · similarity 0.91)
- Detecting and Preventing Hallucinations in Large Vision Language Models (AAAI 2024 · similarity 0.91)
- Scalable Token-Level Hallucination Detection in Large Language Models (NeurIPS 2026 · similarity 0.90)
- HaDeMiF: Hallucination Detection and Mitigation in Large Language Models (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.