Research map / Large language models / Language, safety and interpretability
Semantic Density: Uncertainty Quantification for Large Language Models through Confidence Measurement in Semantic Space
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes semantic density, a response-wise, off-the-shelf confidence metric that weights NLI-based semantic distances to sampled reference responses by their LLM sequence probabilities, outperforming six uncertainty baselines on seven LLMs and four QA benchmarks.
Problem
Existing LLM uncertainty quantification methods are restricted to classification, need extra training or labels, use only lexical information, or are prompt-wise rather than response-wise, so they cannot give each free-form response a trustworthiness score.
Approach
Treats sampled responses as a probability distribution in a semantic space and estimates a response's density via a kernel over semantic distances (derived from NLI class probabilities) weighted by sequence probabilities, yielding a response-wise confidence in [0,1] with no training.
Key results
- Semantic density achieves best AUROC in 26 of 28 model-dataset configurations on four QA benchmarks, versus semantic entropy and other baselines.
- On CoQA, SD reaches AUROC 0.783 to 0.791 for Llama-2-13B, Llama-2-70B and Mixtral-8x22B, compared with 0.614 to 0.633 for semantic entropy.
- Performance remains strong with as few as four reference responses, with only minor decreases compared to ten.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use semantic density instead of existing uncertainty metrics | LLM response confidence estimation · CoQA, TriviaQA, SciQ, and NQ | AUROC | Mixed |
| Use semantic density instead of semantic entropy | LLM response confidence estimation · DUC-2004 summarization | AUROC | Mixed |
| Reduce the number of reference responses from 10 toward 1 (ablation) | Semantic density · CoQA, TriviaQA, SciQ, and NQ | AUROC | Hurt |
| Use at least four reference responses instead of fewer (ablation) | Semantic density · CoQA, TriviaQA, SciQ, and NQ | AUROC | Helped |
| Vary target-response sampling from more greedy to more diverse beam groups (ablation) | Semantic density · CoQA, TriviaQA, SciQ, and NQ | AUROC | No clear effect |
| Use semantic density instead of other metrics as the correctness threshold varies | LLM response confidence estimation · CoQA, TriviaQA, SciQ, and NQ | AUROC | Helped |
Surprising findings
- Degree, which ignores likelihood, beat SD in two cases where the LLM's sequence likelihood was poorly calibrated.
- Semantic entropy variant from a follow-up work performed comparably to or worse than original semantic entropy on several settings.
Limitations
- Requires access to output token probabilities, which proprietary LLMs may not expose.
- Evaluated mainly on sentence-level responses; extension to long-form responses is untested.
- Correctness is judged by Rouge-L threshold against reference answers, which is an imperfect proxy.
Experimental setup
Datasets: CoQA TriviaQA SciQ Natural Questions DUC-2004
Benchmarks: CoQA TriviaQA SciQ NQ DUC-2004
Metrics: AUROC AUPR paired t-test p-value
Baselines: semantic entropy P(True) degree matrix uncertainty length-normalized likelihood length-normalized entropy predictive entropy
Builds on: semantic entropy kernel density estimation diverse beam search Epanechnikov kernel
Closest papers by meaning
- Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities (NeurIPS 2024 · similarity 0.91)
- Efficient semantic uncertainty quantification in language models via diversity-steered sampling (NeurIPS 2025 · similarity 0.91)
- Benchmarking LLMs via Uncertainty Quantification (NeurIPS 2024 · similarity 0.90)
- Improving Uncertainty Estimation through Semantically Diverse Language Generation (ICLR 2025 · similarity 0.89)
- Distinguishing the Knowable from the Unknowable with Language Models (ICML 2024 · similarity 0.89)
- Conformal Prediction with Paraphrase-Aware Scoring for LLM Uncertainty Quantification (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.