Research map / Large language models / Language, safety and interpretability

Integrating Local and Global Entropy for Uncertainty Quantification in LLMs

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Uncertainty quantification for LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Global-Local Uncertainty (GLU), a single-pass, unsupervised UQ score that multiplicatively fuses hidden-state Rényi-2 geometric entropy (global) with top-k token-level Shannon/evidential entropy (local) for LLM response reliability.

Problem

Hallucination detection needs uncertainty scores that are unsupervised, single-pass, length-normalized and architecture-agnostic; existing token-level methods miss confident-but-wrong responses, while hidden-state methods underuse geometry.

Approach

Measure response-level uncertainty as the length-normalized collision (Rényi-2) entropy of the Gram matrix of hidden-state trajectories averaged over layers, and use it as a multiplicative amplifier (1+S) of a local token-level entropy-times-evidential-uncertainty score.

Key results

• GLU achieves the highest mean AUROC (0.676) and mean PRR (0.369) across 18 model-dataset settings, ranking first among 14 methods; RAUQ averages 0.648 AUROC and LogTokU 0.574.
• GLU gives best AUROC in 13 of 18 settings; on Gemma-TriviaQA it reaches 0.817 vs 0.597 for LogTokU and 0.700 for RAUQ.
• Additive fusion (Add-S̃) drops to rank 13 (0.621 AUROC), suggesting multiplicative interaction drives the gain.

Surprising findings

• Additive fusion of the same signals performs far worse than multiplicative fusion, dropping to rank 13 of 14 methods.
• Several simple ablations (e.g., GLU-EU, GLU-EU-SP) collapse to near-chance AUROC, showing the Shannon term is needed.

Limitations

• Requires access to hidden states and token-level output distributions, limiting use to open-weight models.
• Relative strength of global and local signals varies by task; RAUQ remains competitive in some settings where local signal is weak.
• Evaluated only on three models and modest-size benchmarks; the paper's own limitations call for larger models and more languages.

Experimental setup

Datasets: TruthfulQA TriviaQA MATH ArabicaQA LongForm GSM8K multi-turn (lost_in_conversation)

Benchmarks: TruthfulQA TriviaQA MATH ArabicaQA LongForm GSM8K multi-turn

Metrics: AUROC PRR

Baselines: LogProb LogTokU P(True) RAUQ Probe(EOS) Probe(EU) Probe(AVG) RespEntropy Prob(Exact)

Builds on: RAUQ LogTokU evidential deep learning linear representation hypothesis

Closest papers by meaning

• Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities (NeurIPS 2024 · similarity 0.91)
• Inv-Entropy: A Fully Probabilistic Framework for Uncertainty Quantification in Language Models (NeurIPS 2025 · similarity 0.90)
• Distinguishing the Knowable from the Unknowable with Language Models (ICML 2024 · similarity 0.89)
• Conformal Prediction with Paraphrase-Aware Scoring for LLM Uncertainty Quantification (NeurIPS 2026 · similarity 0.89)
• SPANUQ: Span-Level Uncertainty Quantification for Large Language Model Generation (NeurIPS 2026 · similarity 0.88)
• Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling (ICML 2024 · similarity 0.88)