Research map / Large language models / Retrieval and knowledge
ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes internal mechanisms (Knowledge FFNs and Copying Heads) behind RAG hallucinations, proposes ReDeEP, a regression-based detector over decoupled external-context and parametric-knowledge scores, and AARF, a training-free intervention to reduce hallucinations.
Problem
Detecting RAG hallucinations, where outputs conflict with accurate retrieved context, requires disentangling how LLMs use external context versus parametric knowledge; existing detectors either rely on one mechanism or mix them without decoupling.
Approach
Uses mechanistic interpretability to define an External Context Score (attention heads, via copying-head analysis) and a Parametric Knowledge Score (FFN layers via LogitLens JSD), then regresses these decoupled covariates to predict hallucinations; AARF reweights copying heads and knowledge FFNs at inference.
Key results
- On RAGTruth with LLaMA2-7B, ReDeEP(chunk) reaches AUC 0.7458 and F1 0.7190, versus the best baseline AUC 0.7290 (RAGAS) and F1 0.7066 (ChainPoll).
- On Dolly (AC) with LLaMA2-7B, ReDeEP(chunk) reaches AUC 0.7949 and PCC 0.5136, above the best baseline AUC 0.7110 (Trulens).
- AARF reduces hallucinations relative to the baseline model in GPT-4o pairwise comparisons, though the abstract-level gains are described only qualitatively in the text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ReDeEP(chunk) instead of ReDeEP(token) (ablation) | ReDeEP · RAGTruth and Dolly (AC) across LLaMA2-7B and LLaMA2-13B | AUC: 0.7325 -> 0.7458; 0.6884 -> 0.7949; 0.8181 -> 0.8244; 0.7226 -> 0.8420 | Helped |
| Use ReDeEP(chunk) instead of ReDeEP(token) (ablation) | ReDeEP · RAGTruth and Dolly (AC) with LLaMA3-8B | AUC: 0.7522 -> 0.7285; 0.6701 -> 0.7354 | Mixed |
| Use ReDeEP(chunk) instead of ReDeEP(token) (ablation) | ReDeEP · RAGTruth and Dolly (AC) across LLaMA2-7B, LLaMA2-13B, and LLaMA3-8B | F1: 0.6986 -> 0.7190; 0.7244 -> 0.7833; 0.7494 -> 0.7587; 0.7154 -> 0.7603; 0.7132 -> 0.6947; 0.6901 -> 0.7100 | Mixed |
| Increase Copying Head contributions and reduce Knowledge FFN contributions during generation (AARF) | LLaMA models · RAGTruth and Dolly (AC) | truthfulness (GPT-4o pairwise preference) | Helped |
| Increase attention-head contributions and amplify FFN contributions in causal intervention | LLaMA2-7B · RAGTruth truthful dataset | NLL difference | Mixed |
Surprising findings
- Most hallucinations stem from the LLM losing information that copying heads attended to during generation, not from heads failing to attend to the correct context (about 77.5% of attention heads attended correctly).
Limitations
- Proxy metrics (ECS, PKS) are admitted to be imprecise due to LLM complexity.
- Evaluation limited to LLaMA-family models and two RAG hallucination datasets.
- Llama3-8B and Dolly (AC) hallucination labels were produced by manual annotation by three annotators, a small-scale labeling process.
Experimental setup
Datasets: RAGTruth Dolly (AC)
Benchmarks: RAGTruth Dolly (AC)
Metrics: AUC PCC Recall F1 pairwise truthfulness win rate
Baselines: SelfCheckGPT Perplexity LN-Entropy Energy Focus Prompt LMvLM ChainPoll RAGAS Trulens RefCheck P(True)
Builds on: Copying heads (OV circuit eigenvalue analysis) LogitLens ITI SEP ChainPoll
Closest papers by meaning
- LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.93)
- Scalable Token-Level Hallucination Detection in Large Language Models (NeurIPS 2026 · similarity 0.92)
- HaDeMiF: Hallucination Detection and Mitigation in Large Language Models (ICLR 2025 · similarity 0.92)
- Zero-resource Hallucination Detection for Text Generation via Graph-based Contextual Knowledge Triples Modeling (AAAI 2025 · similarity 0.91)
- In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024 · similarity 0.91)
- Entropy Distribution as a Fingerprint for Hallucinations in Generative Models (NeurIPS 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.