Research map / Large language models / AI and society

Position: Uncertainty Quantification Needs Reassessment for Large Language Model Agents

ICML 2025 · Large language models › AI and society · approach: Critiques of LLM claims and judging · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Position paper arguing that the aleatoric/epistemic uncertainty dichotomy is ill-suited to interactive LLM agents, and proposing underspecification uncertainties, interactive learning, and output uncertainties as new research directions.

Problem

Traditional aleatoric vs. epistemic uncertainty definitions conflict with each other and break down in multi-turn, interactive LLM agent settings where users underspecify tasks and inputs.

Approach

Reviews conflicting schools of thought on aleatoric and epistemic uncertainty (even in toy examples) and argues that uncertainty in chat should be framed around underspecification, resolved via interactive follow-up questions, and communicated through language rather than scalar numbers.

Key results

• Reports that GPT-3.5-Turbo-16k detects ambiguous questions with only 57% accuracy (50% random), citing prior work.
• Cites that ambiguities appear in 56% of Natural Questions test questions, citing prior work.
• Cites near-perfect rank correlations (0.8 to 0.999) between aleatoric and epistemic estimators across methods, citing prior work.

Surprising findings

• Aleatoric estimators are about as predictive for OOD detection as epistemic estimators, and epistemic estimators are as predictive of annotator noise as aleatoric ones.
• Human annotators rate only 53% of generated follow-up questions as helpful in resolving ambiguity.

Limitations

• Position paper with no new experiments; claims rest on literature review and toy examples.
• Proposed research directions are not operationalized or evaluated.

Experimental setup

Datasets: Natural Questions

Builds on: active learning conformal prediction mutual-information epistemic uncertainty decomposition

Closest papers by meaning

• Uncertainty-Aware Clarification in LLM Agents with Information Gain (ICML 2026 · similarity 0.90)
• Large Language Models Must Be Taught to Know What They Don’t Know (NeurIPS 2024 · similarity 0.90)
• Do LLMs estimate uncertainty well in instruction-following? (ICLR 2025 · similarity 0.89)
• To Believe or Not to Believe Your LLM: Iterative Prompting for Estimating Epistemic Uncertainty (NeurIPS 2024 · similarity 0.89)
• Narrowing the Collaboration Gap, Probably (NeurIPS 2026 · similarity 0.88)
• Localizing Input Uncertainty Quantification for Large Language Models via Shapley Values (NeurIPS 2026 · similarity 0.88)