Research map / Large language models / Language, safety and interpretability
Task-Awareness Improves LLM Generations and Uncertainty
Tim Tomov, Dominik Fuchsgruber, Stephan Günnemann
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| decode with latent-space Bayes-optimal response instead of beam search | LLM decoding · MAQA sets (block 1) | Hamming distance: 0.892 -> 0.729 | Improved |
| decode with Bayes-optimal response instead of lowest-risk sample l_Sample | LLM decoding · MAQA sets (block 1) | Hamming distance: 0.838 -> 0.729 | Improved |
| decode with Bayes-optimal response instead of majority-vote l_MAP | LLM decoding · MAQA sets (block 3) | Hamming distance: 0.913 -> 0.858 | Improved |
| decode with Bayes response instead of beam search | LLM decoding · CNN/DailyMail summarization (block 1) | structural Hamming: 2.176 -> 1.603 | Improved |
| decode with Bayes response instead of beam search | LLM decoding · MAQA probability simplex (block 2) | KL divergence: 7.792 -> 5.632 | Improved |
| decode with Bayes response instead of beam search | LLM decoding · WMT19 FI-EN (block 1) | cosine distance: 0.131 -> 0.124 | Improved |
| decode with Bayes response instead of beam search | LLM decoding · TriviaQA classification (block 1) | exact match: 0.627 -> 0.632 | No clear effect |
| increase number of MC samples for Bayes-optimal decoding | l_Bayes · MAQA sets | Hamming distance | Improved |
| use Bayes risk as uncertainty instead of semantic entropy and other estimators | Bayes risk uncertainty · 4 of 5 latent structures | PRR | Improved |
| use Bayes risk uncertainty on single-label classification | Bayes risk uncertainty · TriviaQA classification | PRR | No clear effect |
Closest papers by meaning
- Graph-based Uncertainty Metrics for Long-form Language Model Generations (NeurIPS 2024 · similarity 0.89)
- Uncertainty-Aware Decoding with Minimum Bayes Risk (ICLR 2025 · similarity 0.88)
- Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.88)
- Large Language Models Must Be Taught to Know What They Don’t Know (NeurIPS 2024 · similarity 0.88)
- Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs (ICLR 2024 · similarity 0.88)
- XTC: Head-Aware Sampling by Excluding Top Choices (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
