AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

How do LLMs Compute Verbal Confidence?

Dharshan Kumaran, Arthur Conmy, Federico Barbero, Simon Osindero, Viorica Patraucean, Petar Veličković

ICML 2026 · Large language models › Language, safety and interpretability · approach: Uncertainty quantification for LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
steer activations at PANL with high/low confidence vectorsGemma 3 27B · TriviaQAverbal confidenceImproved
steer at answer-colon (AC) positionGemma 3 27B · TriviaQAverbal confidenceNo clear effect
patch clean PANL activation after answer corruption (ablation)Gemma 3 27B · TriviaQA high-confidenceconfidence recoveryImproved
patch clean PANL+1 activation (control) (ablation)Gemma 3 27B · TriviaQA high-confidenceconfidence recoveryNo clear effect
mean-ablate PANL or CC residual (ablation)Gemma 3 27B · TriviaQAverbal confidence disruptionHurt
cross-confidence activation swap at PANL (ablation)Gemma 3 27B · TriviaQAconfidence changeImproved
block attention from CC to question and answer tokens (ablation)Gemma 3 27B · TriviaQA categoricalfirst-token change rate: ~10% change rateNo clear effect
block attention from CC to PANL (minimal numeric prompt) (ablation)Gemma 3 27B · TriviaQA numeric 0-9first-token change rate: ~21% (layers 30-36)Hurt
block attention from CC to PANL (categorical prompt) (ablation)Gemma 3 27B · TriviaQA categoricalfirst-token change rateNo clear effect
block attention from PANL to answer tokens (ablation)Gemma 3 27B · TriviaQA numeric 0-9first-token change rate: ~20% (layers 22-28)Hurt
add PANL activations to log-probability predictors of verbal confidenceGemma 3 27B · TriviaQAvariance explained: 10.0% -> 38.0%Improved
mean-ablate PANL in reasoning model (ablation)Magistral Small 24B · TriviaQAfirst-token change rateNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy