Research map / Large language models / Language, safety and interpretability
In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Activation Decoding, a constrained decoding method that penalizes next-token candidates whose activations over the in-context prompt tokens have high entropy (low "in-context sharpness"), to reduce hallucinations.
Problem
LLMs frequently hallucinate factual errors, and it is unclear what in their inner representations signals correctness; existing mitigation often needs external knowledge, fine-tuning, or labeled data.
Approach
Correct generations show sharper (lower-entropy) activations over in-context tokens in intermediate layers; the paper defines a contextual entropy metric by projecting hidden states to vocabulary via the LM head, normalizing over prompt positions, and uses it as a penalty term in decoding, with entropies precomputed per vocabulary token.
Key results
- On open-ended TruthfulQA, Activation Decoding improves Truth*Info by 8.6 points over greedy decoding for LLaMA2-70B-chat (55.7 vs 47.1).
- On HotpotQA F1, LLaMA2-13B-chat gains 4.7 points over greedy decoding (26.4 vs 21.7), with gains up to 4.8 F1 on TriviaQA and 3.7 on NQ across model sizes.
- Contextual entropy achieves AUROC above 0.75 for distinguishing correct from incorrect answers on GF-CFT at the 26th and 28th layers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add contextual entropy penalty during decoding | LLaMA2-chat models · TruthfulQA open-ended | Truth*Info: 55.8 -> 59.1; 57.5 -> 62.3; 47.1 -> 55.7 | Helped |
| Add contextual entropy penalty during decoding | LLaMA2-7B-chat · TriviaQA | F1 score: 44.3 -> 46.4 | Helped |
| Add contextual entropy penalty during decoding | LLaMA2-13B-chat · HotpotQA | F1 score: 21.7 -> 26.4 | Helped |
| Add contextual entropy penalty during decoding | LLaMA2-70B-chat · Natural Questions | F1 score: 34.1 -> 37.8 | Helped |
| Combine Activation Decoding with Dola | LLaMA2-chat · Knowledge-seeking datasets | F1 score: TriviaQA 7B: 46.4 -> 46.5; HotpotQA 13B: 26.4 -> 26.2; NQ 70B: 37.8 -> 37.6 | Mixed |
| Add Activation Decoding to LLaMA2-70B-chat | LLaMA2-70B-chat · TruthfulQA open-ended | %Truth: 68.8 -> 65.7 | Hurt |
| Add Activation Decoding to LLaMA2-70B-chat | LLaMA2-70B-chat · TruthfulQA open-ended | %Info: 78.3 -> 90.0 | Helped |
| Use entropy score alone for classification (ablation) | LLaMa-2-7B · TruthfulQA multiple-choice | MC2: 43.4 -> 61.1 | Helped |
| Use entropy score alone for classification (ablation) | LLaMa-2-7B · TruthfulQA multiple-choice | MC1: 28.5 -> 27.1 | Hurt |
| Use the 26th informative layer instead of the 24th (ablation) | LLaMa-2-7B · TruthfulQA multiple-choice | MC2: 46.9 -> 45.3 | Hurt |
Surprising findings
- Entropy-based decoding with the proposed method reduces inference time by 7.3% relative to DoLa while adding 23.4% over greedy decoding, so it is cheaper than DoLa despite the extra computation.
- Representation-based corrections can introduce new errors, and performance gains vary by dataset with a ceiling.
Limitations
- Only addresses model-related hallucinations; cannot fix errors requiring external or outdated knowledge.
- Hyperparameters (layer, lambda) must be tuned per model and benchmark, and the paper reports gains are sensitive to choices and sometimes drop Truth.
- Experiments are limited to LLaMA-2-chat models and mainly to short-form QA, and the case study uses a small hand-annotated dataset.
Experimental setup
Datasets: CounterFact TruthfulQA TriviaQA HotpotQA Natural Questions
Benchmarks: TruthfulQA TriviaQA HotpotQA Natural Questions CounterFact (GF-CFT, Raw-CFT for AUROC analysis)
Metrics: Truth*Info %Truth %Info %Reject MC1 MC2 MC3 Exact Match F1 AUROC
Baselines: Greedy decoding DoLa ITI Self-eval
Builds on: DoLa Inference-time Intervention (ITI) logit lens projection
Closest papers by meaning
- Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations (NeurIPS 2025 · similarity 0.92)
- Attributive Reasoning for Hallucination Diagnosis of Large Language Models (AAAI 2025 · similarity 0.92)
- Scalable Token-Level Hallucination Detection in Large Language Models (NeurIPS 2026 · similarity 0.92)
- INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection (ICLR 2024 · similarity 0.91)
- CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models (NeurIPS 2024 · similarity 0.91)
- HaDeMiF: Hallucination Detection and Mitigation in Large Language Models (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.