Research map / Large language models / Language, safety and interpretability
Uncertainty as Feature Gaps: Epistemic Uncertainty Quantification of LLMs in Contextual Question-Answering
Yavuz Faruk Bakman, Sungmin Kang, Zhiqi Huang, Duygu Nur Yaldiz, Catarina G Belém, Chenyang Zhu, Anoop Kumar, Alfy Samuel, Daben Liu, Salman Avestimehr, Sai Praneeth Karimireddy
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add contrastive-difference feature directions instead of Positive-PCA (ablation) | Feature-Gaps · HotpotQA (Llama3.1-8B) | PRR: 47.1 -> 66.6 | Improved |
| add contrastive-difference feature directions instead of Negative-PCA (ablation) | Feature-Gaps · Qasper (Llama3.1-8B) | PRR: 40.5 -> 64.9 | Improved |
| add feature-finding instead of random directions (ablation) | Feature-Gaps · NarrativeQA (Mistral-7B) | PRR: 7.6 -> 38.5 | Improved |
| add contrastive feature extraction instead of unpaired All-PCA (ablation) | Feature-Gaps · Qasper (Qwen2.5-7B) | PRR: -4.3 -> 58.5 | Improved |
| use extracted feature directions instead of supervised Mean-Diff (ablation) | Feature-Gaps · HotpotQA (Llama3.1-8B) | PRR: 53.1 -> 66.6 | Improved |
| use extracted feature directions instead of supervised Mean-Diff (ablation) | Feature-Gaps · Qasper (Qwen2.5-7B) | PRR: 57.3 -> 58.5 | No clear effect |
| add ensemble of three features instead of best single feature (ablation) | Feature-Gaps · Qasper (Llama3.1-8B) | PRR: 62.0 -> 64.9 | Improved |
| add ensemble of three features instead of best single feature (ablation) | Feature-Gaps · NarrativeQA (Mistral-7B) | PRR: 37.3 -> 38.5 | Improved |
| add ensemble of three features instead of best single feature (ablation) | Feature-Gaps · NarrativeQA (Qwen2.5-7B) | PRR: 56.9 -> 51.0 | Hurt |
| increase labeled samples from 64 to 256 (ablation) | Feature-Gaps · Qasper (Mistral-7B) | PRR: 38.3 -> 59.5 | Improved |
| increase labeled samples from 128 to 256 (ablation) | Feature-Gaps · Qasper (Llama3.1-8B) | PRR: 63.2 -> 64.9 | No clear effect |
| increase labeled samples from 64 to 256 (ablation) | Feature-Gaps · NarrativeQA (Llama3.1-8B) | PRR: 57.5 -> 59.7 | No clear effect |
Closest papers by meaning
- To Believe or Not to Believe Your LLM: Iterative Prompting for Estimating Epistemic Uncertainty (NeurIPS 2024 · similarity 0.90)
- Efficient semantic uncertainty quantification in language models via diversity-steered sampling (NeurIPS 2025 · similarity 0.89)
- FLUE: Streamlined Uncertainty Estimation for Large Language Models (AAAI 2025 · similarity 0.88)
- Semantic Density: Uncertainty Quantification for Large Language Models through Confidence Measurement in Semantic Space (NeurIPS 2024 · similarity 0.88)
- A Semantic-Sampling Framework for Evaluating Calibration in Open-Ended Question Answering (NeurIPS 2026 · similarity 0.88)
- Argument Graph Uncertainty: Quantifying Uncertainty from the Logical Structure of Reasoning Chains (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
