Research map / Large language models / Language, safety and interpretability
ConfTuner: Training Large Language Models to Express Their Confidence Verbally
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ConfTuner, a LoRA fine-tuning method that trains LLMs to verbalize calibrated confidence using a tokenized Brier score (proved a proper scoring rule), without ground-truth confidence or proxy labels.
Problem
LLMs are overconfident when verbalizing confidence; existing calibration methods rely on prompting or heuristic proxy confidence targets (group accuracy, consistency, model judgment), which are limited in effectiveness and generalizability.
Approach
Extend the Brier score to token distributions over confidence tokens: the loss is the expected squared error between each confidence token's value and the correctness label under the model's softmax over confidence tokens; a theorem shows its minimizer puts all mass on the token closest to the true correctness probability.
Key results
- ConfTuner on LLaMA reduces HotpotQA ECE from 0.4803 (Base) to 0.0405 and averages ECE 0.1082 vs 0.2768 for Base across five datasets.
- ConfTuner raises average AUROC to 0.6740 (LLaMA), 0.6861 (Qwen), and 0.6810 (Ministral), outperforming the baselines in most settings.
- A ConfTuner LLaMA model trained to calibrate GPT-4o responses lowers average ECE from 0.1640 to 0.0970 and raises average AUROC from 0.5945 to 0.6275.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune with ConfTuner instead of LACIE | Llama-3.1-8B-Instruct · HotpotQA | ECE: 0.2954 -> 0.0405 | Helped |
| Fine-tune with ConfTuner instead of SaySelf | Llama-3.1-8B-Instruct · GSM8K | AUROC: 0.5425 -> 0.7007 | Helped |
| Fine-tune with ConfTuner instead of SaySelf | Llama-3.1-8B-Instruct · TruthfulQA | AUROC: 0.5890 -> 0.5739 | Hurt |
| Use confidence tokens 0%-100% instead of 0-9 (ablation) | ConfTuner on LLaMA · HotpotQA | ECE: 0.0605 -> 0.0405 | Helped |
| Use confidence tokens 0%-100% instead of 0-9 (ablation) | ConfTuner on LLaMA · HotpotQA | AUROC: 0.7248 -> 0.7383 | Helped |
| Generate the answer before the confidence (ablation) | ConfTuner on LLaMA · HotpotQA | AUROC: 0.7263 -> 0.7383 | Helped |
| Generate the answer before the confidence (ablation) | ConfTuner on LLaMA · HotpotQA | ECE: 0.0685 -> 0.0405 | Helped |
| Train on GSM8K instead of HotpotQA (ablation) | ConfTuner on LLaMA · GSM8K | ECE: 0.1276 -> 0.0753 | Helped |
| Train on GSM8K instead of HotpotQA (ablation) | ConfTuner on LLaMA · HotpotQA | ECE: 0.0405 -> 0.2308 | Hurt |
| Train on GSM8K instead of HotpotQA (ablation) | ConfTuner on LLaMA · StrategyQA | AUROC: 0.6750 -> 0.6826 | Helped |
Surprising findings
- Outputting confidence before the answer yields markedly worse calibration than outputting it after the answer, suggesting answer hidden states carry confidence information.
- A simple classifier head on frozen hidden states performs worse than end-to-end ConfTuner fine-tuning.
Limitations
- Generalization to complex conversational contexts and more diverse confidence expressions is untested.
- Practical calibration depends on data quality, architecture and optimization dynamics beyond the theoretical guarantee.
- Uses a regularization SFT term for LLaMA; without it LLaMA sometimes omits confidence or repeats text.
Experimental setup
Datasets: HotpotQA GSM8K
Benchmarks: HotpotQA GSM8K TriviaQA StrategyQA TruthfulQA
Metrics: ECE AUROC accuracy
Baselines: Base Ensemble SaySelf LACIE P(True)
Builds on: Brier score SaySelf LACIE
Closest papers by meaning
- Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs (ICLR 2024 · similarity 0.90)
- SteerConf: Steering LLMs for Confidence Elicitation (NeurIPS 2025 · similarity 0.90)
- Learning to Route LLMs with Confidence Tokens (ICML 2025 · similarity 0.89)
- Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator (NeurIPS 2025 · similarity 0.89)
- Calibrating LLMs with Semantic-level Reward (NeurIPS 2026 · similarity 0.89)
- Large Language Models Must Be Taught to Know What They Don’t Know (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.