Research map / Large language models / Language, safety and interpretability

Steer LLM Latents for Hallucination Detection

ICML 2025 · Large language models › Language, safety and interpretability · approach: Task vectors and steering · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Truthfulness Separator Vector (TSV), a single learned steering vector added to an intermediate LLM layer at inference, trained with a few labeled exemplars plus optimal-transport pseudo-labeling of unlabeled generations to separate truthful from hallucinated answers for detection.

Problem

LLM latent embeddings optimized for fluency do not clearly separate truthful from hallucinated generations, and labeled hallucination data is costly to obtain, so detection from internal states is weak.

Approach

Learn a lightweight steering vector added to intermediate-layer hidden states (model weights frozen) that makes last-token embeddings cluster around truthful/hallucinated vMF class prototypes; augment training with unlabeled generations pseudo-labeled via Sinkhorn optimal transport and confidence-based selection.

Key results

• On LLaMA-3.1-8B, TSV reaches 84.2 AUROC on TruthfulQA versus 70.6 for HaloScope (single sampling), and 85.5 for the fully supervised TSV upper bound.
• On Qwen-2.5-7B, TSV achieves 87.3 AUROC on TruthfulQA and 82.0 on SciQ, outperforming all unsupervised baselines in the main table.
• Ablation shows adding TSV improves TruthfulQA AUROC by 28.7 points over the initial-training-only variant without TSV (52.2 -> 80.9 for IT-only with TSV).

Measured effects reported in the paper

Surprising findings

• Without the steering vector, the pseudo-labeling augmentation gives no improvement (AUROC stays near 52%), suggesting augmentation only helps when representations are already well structured.
• TSV with only 32 labeled examples nearly matches a fully supervised upper bound trained on all unlabeled data with ground-truth labels (84.2 vs 85.5 on TruthfulQA).

Limitations

• Focuses on sentence-level, short-form QA; fine-grained span-level hallucination detection is not addressed.
• Pseudo-label noise and class distribution mismatch degrade performance (e.g., 84.2 to 81.3 AUROC at 25% label noise).
• Performance on the OOD transfer (e.g., TriviaQA from TruthfulQA at 79.8) is lower than in-distribution results.

Experimental setup

Datasets: TruthfulQA TriviaQA SciQ NQ Open

Benchmarks: TruthfulQA TriviaQA SciQ NQ Open

Metrics: AUROC

Baselines: Perplexity LN-Entropy Semantic Entropy Lexical Similarity EigenScore SelfCKGPT Verbalize Self-evaluation CCS HaloScope SAPLMA LoRA

Builds on: HaloScope SAPLMA Sinkhorn algorithm LoReFT LoRA

Closest papers by meaning

• LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.92)
• HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detection (NeurIPS 2024 · similarity 0.92)
• Scalable Token-Level Hallucination Detection in Large Language Models (NeurIPS 2026 · similarity 0.92)
• Sample Transform Cost-Based Training-Free Hallucination Detector for Large Language Models (NeurIPS 2026 · similarity 0.91)
• Steer Where It Matters: Token-Level Visual-Sensitivity Steering for LVLMs Hallucination Mitigation (ICML 2026 · similarity 0.91)
• Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViT (NeurIPS 2025 · similarity 0.91)