Research map / Large language models / Language, safety and interpretability
Calibrating LLMs with Information-Theoretic Evidential Deep Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IB-EDL, which regularizes evidential deep learning with an information bottleneck on pre-evidence to calibrate fine-tuned LLMs in a single forward pass.
Problem
Fine-tuned LLMs, especially on small datasets, are overconfident and poorly calibrated, and EDL-based uncertainty estimates can still be overconfident because the training objective concentrates Dirichlet distributions.
Approach
Choose the pre-evidence as the IB stochastic variable with a standard Gaussian prior, so the IB objective is a valid lower bound on I(Z,Y) without learning an approximate decoder; the KL term acts as an l2 penalty on evidence magnitude, suppressing overly large evidence and overconfidence.
Key results
- On fine-tuned Llama3-8B, IB-EDL reduces ECE on ARC-C to 2.78% vs 19.68% for MAP while keeping accuracy comparable (81.14% vs 79.74%).
- On OOD detection (OBQA to ARC-C, Llama2-7B), IB-EDL reaches 87.47 AUROC (MP) vs 76.07 for MAP.
- Under 30% label noise, IB-EDL achieves the highest accuracy among EDL methods on most datasets, e.g., 80.27% on ARC-E for Llama2-7B vs 76.16% for EDL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add information-bottleneck regularization to EDL | Llama2-7B · ARC-C, ARC-E, OBQA, CSQA, SciQ, and RACE | ECE | Helped |
| Add information-bottleneck regularization to EDL | Llama2-7B · ARC-C, ARC-E, OBQA, CSQA, SciQ, and RACE | NLL | Helped |
| Add information-bottleneck regularization to EDL | Llama3-8B · ARC-C, ARC-E, OBQA, and CSQA | ECE | Helped |
| Add information-bottleneck regularization to EDL | Llama3-8B · SciQ and RACE | ECE | Mixed |
| Add information-bottleneck regularization to EDL | Llama3-8B · ARC-C, ARC-E, OBQA, CSQA, SciQ, and RACE | accuracy | No clear effect |
| Add information-bottleneck regularization to EDL | Llama2-7B and Llama3-8B · OBQA → ARC-C, ARC-E, and CSQA | AUROC | Helped |
| Add information-bottleneck regularization to EDL | Llama2-7B · ARC-C, ARC-E, OBQA, CSQA, SciQ, and RACE with 30% noisy labels | accuracy | Helped |
| Add information-bottleneck regularization to EDL | Llama3-8B · ARC-C, ARC-E, OBQA, CSQA, SciQ, and RACE with 30% noisy labels | accuracy | Mixed |
| Increase the number of pre-evidence samples K (ablation) | IB-EDL · ARC-C, Llama2-7B | accuracy | Helped |
| Increase the number of pre-evidence samples K (ablation) | IB-EDL · ARC-C, Llama2-7B | ECE and NLL | No clear effect |
| Vary the regularization weight β (ablation) | IB-EDL · ARC-C, Llama2-7B | ECE and NLL: 40%; 50% | Helped |
| Add a post-hoc adjustment α_j ← α_j − ζ·σ_j (ablation) | IB-EDL · OOD calibration experiments | calibration | Helped |
Surprising findings
- Vanilla MAP fine-tuning yields very high ECE (e.g., ~30% on ARC-C with Llama2-7B), showing severe overconfidence.
- I-EDL and R-EDL show inconsistent gains; R-EDL attains the best ARC-C accuracy for Llama2-7B but has high ECE on some datasets.
Limitations
- Assumes pre-evidences are uncorrelated when predicting the covariance, which may be restrictive.
- Evaluation limited to classification tasks with multiple-choice options; generative tasks untested.
- Requires per-dataset beta tuning via grid search.
Experimental setup
Datasets: ARC-C ARC-E OpenbookQA CommonsenseQA SciQ RACE
Benchmarks: ARC-C ARC-E OpenbookQA CommonsenseQA SciQ RACE
Metrics: accuracy ECE NLL AUROC
Baselines: MAP MC-Dropout Deep Ensemble Laplace-LoRA EDL VID I-EDL R-EDL
Builds on: Evidential Deep Learning Information Bottleneck Variational Information Bottleneck I-EDL R-EDL
Closest papers by meaning
- Uncertainty Estimation by Flexible Evidential Deep Learning (NeurIPS 2025 · similarity 0.91)
- Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage? (NeurIPS 2024 · similarity 0.90)
- Uncertainty Estimation by Density Aware Evidential Deep Learning (ICML 2024 · similarity 0.87)
- The Evidence Contraction Issue in Deep Evidential Regression: Discussion and Solution (AAAI 2024 · similarity 0.86)
- Large Language Models Must Be Taught to Know What They Don’t Know (NeurIPS 2024 · similarity 0.86)
- Benchmarking LLMs via Uncertainty Quantification (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.