Research map / Large language models / Language, safety and interpretability
Why Are LLMs Confidently Wrong? Correcting Overconfident Errors via Causal Head Intervention
Jing Ren, Bowen Li, Ziqi Xu, Xuechao Yang, Feng Xia
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoCHI, a class-oriented causal head intervention method that targets attention heads causing overconfident LLM errors, rather than adjusting outputs post hoc or retraining.
Problem
LLMs often make overconfident errors, assigning high confidence to incorrect predictions, which undermines reliability; existing calibration methods act at the output level via post-hoc adjustment or retraining without addressing underlying causes.
Approach
Identifies internal attention heads causally responsible for overconfident errors and intervenes on them in a class-oriented manner, addressing miscalibration at its internal source rather than at the output level.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply CoCHI head masking to the model | LLaMA-3.1-8B · MMLU | ECE: 0.0569 (TS) -> 0.0490 (CoCHI) | Improved |
| apply CoCHI head masking to the model | Mistral-7B · MedMCQA | ECE: 0.0604 (TS) -> 0.0383 (CoCHI) | Improved |
| apply CoCHI head masking to the model | Gemma-2-9B · MMLU | accuracy: 0.6653 (TS) -> 0.6730 (CoCHI) | Improved |
| apply CoCHI instead of TS on LLaMA-3.1-8B and Mistral-7B | LLaMA-3.1-8B · OpenBookQA | accuracy: 0.6827 (TS) -> 0.6700 (CoCHI) | Hurt |
| use ITI instead of CoCHI as the calibration method | LLaMA-3.1-8B and Mistral-7B · MMLU | ECE: 0.0490 (CoCHI) vs 0.2014 (ITI) on LLaMA; 0.3693 ITI on Mistral | Hurt |
| mask only wrong-high facilitating heads (ablation) | CoCHI · MMLU | accuracy: 0.6120 -> 0.2044 | Hurt |
| use the broad canonical union mask without top-k selection (ablation) | CoCHI · MMLU | ECE: 0.0490 -> 0.3850 | Hurt |
| add causal head selection instead of random size-matched masking (ablation) | CoCHI · MMLU | ECE: 0.4330 -> 0.0490 | Improved |
| add wrong-high facilitating heads and top-k ranking to the correct-high interfering-only mask (ablation) | CoCHI · MMLU | accuracy: 0.4267 -> 0.6120 | Improved |
| mask only correct-high non-facilitating heads (ablation) | CoCHI · MMLU | ECE: 0.0490 -> 0.1518 | Hurt |
| raise confidence threshold tau_c from 0.5 to 0.7 or 0.9 (ablation) | CoCHI · MMLU | ECE: 0.0490 -> 0.0705 (0.7), 0.0696 (0.9) | Hurt |
| lower confidence threshold tau_c to 0.2 (ablation) | CoCHI · MMLU | accuracy: 0.6120 -> 0.5933 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Thermometer — Thermometer: Towards Universal Calibration for Large Language Models (ICML 2024) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Experimental setup
Baselines: post-hoc calibration methods retraining-based calibration methods
Closest papers by meaning
- Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs (ICLR 2024 · similarity 0.91)
- SteerConf: Steering LLMs for Confidence Elicitation (NeurIPS 2025 · similarity 0.91)
- Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator (NeurIPS 2025 · similarity 0.91)
- Calibrating Large Language Models with Sample Consistency (AAAI 2025 · similarity 0.90)
- Calibrating LLMs with Semantic-level Reward (NeurIPS 2026 · similarity 0.90)
- ConfTuner: Training Large Language Models to Express Their Confidence Verbally (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
