Research map / Large language models / Language, safety and interpretability
Improving Uncertainty Estimation through Semantically Diverse Language Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Semantically Diverse Language Generation (SDLG), an importance-sampling method that steers LLM generation toward semantically diverse yet likely alternatives to estimate semantic entropy for hallucination detection.
Problem
Estimating aleatoric semantic uncertainty in autoregressive LLMs is imprecise with few samples, because multinomial sampling rarely produces semantically diverse yet likely output sequences, and sampling is computationally expensive.
Approach
Uses NLI-model gradients to score tokens by attribution, substitution, and importance, deterministically substitutes the top-ranked token at a word start, regenerates the suffix, and corrects with importance-sampling weights; also derives a proper semantic entropy estimator from importance sampling.
Key results
- On TruthfulQA with OPT-13b, SE with SDLG reaches AUROC .956 (Rouge-L threshold 0.5) versus .453 for SE multinomial sampling with the improper estimator and .775 for SAR.
- On CoQA, SDLG-based semantic entropy is competitive with the best multinomial-sampling variant (e.g., .744 vs .744 for OPT-2.7b), with smaller gains than on TruthfulQA.
- SDLG requires about 15% (TruthfulQA), 33% (CoQA), and 21% (TriviaQA) fewer FLOPs than multinomial sampling-based semantic entropy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SDLG instead of the strongest listed baseline for semantic entropy | OPT language models · TruthfulQA | AUROC: .846 -> .920; .781 -> .881; .896 -> .956; .864 -> .927 | Helped |
| Use SDLG instead of the strongest listed baseline for semantic entropy | OPT language models · CoQA | AUROC: .744 -> .744; .764 -> .759; .758 -> .760; .767 -> .768 | Mixed |
| Use SDLG instead of the strongest listed baseline for semantic entropy | OPT language models · TriviaQA | AUROC: .808 -> .809; .823 -> .829; .841 -> .845; .837 -> .840 | Helped |
| Use the proper semantic entropy estimator rather than the improper estimator | Semantic entropy with multinomial sampling · Question answering tasks | AUROC | Helped |
| Add targeted semantic token substitutions to multinomial sampling | SDLG · CoQA semantic clusters | semantic clusters found: +19%; +74% | Helped |
| Use SDLG rather than multinomial sampling | Uncertainty estimation · TruthfulQA | FLOPs: 15% | Helped |
| Use SDLG rather than multinomial sampling | Uncertainty estimation · CoQA | FLOPs: 33% | Helped |
| Use SDLG rather than multinomial sampling | Uncertainty estimation · TriviaQA | FLOPs: 21% | Helped |
| Include the attribution score in token ranking (ablation) | SDLG · Question answering tasks | AUROC | Helped |
| Include the substitution score in token ranking (ablation) | SDLG · Question answering tasks | AUROC | Helped |
| Include the importance score in token ranking (ablation) | SDLG · Question answering tasks | AUROC | Helped |
Surprising findings
- The improper semantic entropy estimator of prior work performs very poorly on TruthfulQA at some model sizes (e.g., AUROC .405 for OPT-2.7b), while the proper estimator is much stronger.
- The advantage of the method is smallest on CoQA, the dataset with medium-length answers.
Limitations
- Evaluated only on free-form question answering with OPT models up to 30B.
- Assumes each output sequence belongs to a single semantic cluster and that the substituted position is the most probable position to induce semantic change.
- Only estimates aleatoric semantic uncertainty; epistemic semantic uncertainty is not addressed.
Experimental setup
Datasets: TruthfulQA CoQA TriviaQA
Benchmarks: TruthfulQA CoQA TriviaQA
Metrics: AUROC Rouge-L Rouge-1 BLEURT
Baselines: Predictive Entropy Length-Normalized Predictive Entropy SAR SE multinomial sampling SE diverse beam search
Builds on: semantic entropy importance sampling diverse beam search SAR
Closest papers by meaning
- Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities (NeurIPS 2024 · similarity 0.91)
- Forking Paths in Neural Text Generation (ICLR 2025 · similarity 0.91)
- Efficient semantic uncertainty quantification in language models via diversity-steered sampling (NeurIPS 2025 · similarity 0.90)
- Distinguishing the Knowable from the Unknowable with Language Models (ICML 2024 · similarity 0.90)
- To Believe or Not to Believe Your LLM: Iterative Prompting for Estimating Epistemic Uncertainty (NeurIPS 2024 · similarity 0.90)
- Semantic Density: Uncertainty Quantification for Large Language Models through Confidence Measurement in Semantic Space (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.