Research map / Large language models / Language, safety and interpretability

Sample Transform Cost-Based Training-Free Hallucination Detector for Large Language Models

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Statistical frameworks for LLM evaluation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes training-free hallucination detectors AvgWD and EigenWD that derive average and spectral complexity signals from pairwise sample-to-sample optimal-transport (W2) distances over generated-token hidden states, with a teacher-forcing extension to black-box LLMs.

Problem

Detecting hallucinations in LLMs without training or external verification, using the complexity of the prompt-conditioned response distribution rather than single-response uncertainty or text-level self-consistency.

Approach

Represent each sampled response as an empirical measure over its mid-layer generated-token embeddings, compute pairwise W2 optimal-transport distances to form a cost matrix, and summarize it by its mean (AvgWD) and by the spectral complexity of a Gaussian-kernelized version (EigenWD); black-box models are handled by teacher-forcing samples through an open teacher model.

Key results

• EigenWD achieves the highest average white-box AUROC on Llama-3.2-3B (0.743 vs 0.715 best baseline), Llama-2-7B (0.744 vs 0.723), and Llama-3.1-8B (0.735 vs 0.718).
• AvgWD achieves the best average AUROC on Qwen3-8B (0.701 vs 0.691 best baseline) and Qwen3-32B (0.722 vs 0.700).
• In black-box teacher-forcing experiments with DeepSeek-chat and Llama-2-7B teacher on 10 samples, EigenWD averages 0.7743 AUROC vs 0.7318 for the best baseline (ER).

Surprising findings

• AvgWD is stronger than EigenWD on Qwen models, while EigenWD is stronger on Llama models, indicating the two signals capture different hallucination patterns by model family.

Limitations

• Time complexity of pairwise exact OT is higher than baselines.
• Labels for CoQA/SQuAD use ROUGE-L thresholds and for MATH-500/CNN use GPT-4o judgments, which may be noisy.
• Black-box results depend on an accessible open teacher model and are reported mainly in appendix.

Experimental setup

Datasets: CoQA SQuAD MATH-500 CNN/DailyMail SciQ NQ-open

Benchmarks: CoQA SQuAD MATH-500 CNN/DailyMail SciQ NQ-open

Metrics: AUROC

Baselines: Discrete Semantic Entropy Eigenscore Length-Normalized Entropy Lexical Similarity Effective Rank

Builds on: INSIDE (EigenScore) semantic entropy Effective Rank-based uncertainty

Closest papers by meaning

• Scalable Token-Level Hallucination Detection in Large Language Models (NeurIPS 2026 · similarity 0.93)
• Entropy Distribution as a Fingerprint for Hallucinations in Generative Models (NeurIPS 2026 · similarity 0.93)
• HaDeMiF: Hallucination Detection and Mitigation in Large Language Models (ICLR 2025 · similarity 0.92)
• LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.92)
• Detecting and Preventing Hallucinations in Large Vision Language Models (AAAI 2024 · similarity 0.91)
• Steer LLM Latents for Hallucination Detection (ICML 2025 · similarity 0.91)