Research map / Large language models / Retrieval and knowledge
CrAM: Credibility-Aware Attention Modification in LLMs for Combating Misinformation in RAG
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CrAM, a training-free plug-and-play method that ranks attention heads by their indirect effect on misinformation-driven wrong answers and scales down attention to low-credibility document tokens, improving RAG QA accuracy under misinformation.
Problem
RAG systems are misled by misinformation in retrieved documents; existing credibility-aware RAG fixes require supervised fine-tuning, and discarding low-credibility documents loses useful information.
Approach
Identify influential attention heads via a causal-tracing-style indirect effect measured on misinformation-induced wrong answers, then at inference rescale attention weights to each document's tokens by its normalized credibility score (no weight updates).
Key results
- Under ideal credibility scores with 4 high-credibility plus 1 misinformation document, CrAM raises Llama2-13B TriviaQA EM from 28.00 (Naive RAG) to 59.90, a +31.9 point gain over the second-best method.
- CrAM outperforms the SFT-based CAG-13B in most settings on NQ and TriviaQA with 1 to 3 misinformation documents, per the paper's F1 comparison.
- Under GPT-generated credibility scores, CrAM beats Prompt Based and Exclusion on all three LLMs and both datasets (e.g., Llama3-8B TriviaQA EM 62.20 vs 57.70 for Exclusion).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply CrAM instead of the strongest reported non-SFT baseline under ideal credibility scores | Qwen1.5-7B · NQ and TriviaQA, 4 high-credibility + 1 low-credibility document | EM: NQ: 12.20 -> 29.10; TriviaQA: 27.40 -> 52.90 | Helped |
| Apply CrAM instead of the strongest reported non-SFT baseline under ideal credibility scores | Llama2-13B · NQ and TriviaQA, 4 high-credibility + 1 low-credibility document | EM: NQ: 12.50 -> 33.60; TriviaQA: 23.10 -> 59.90 | Helped |
| Apply CrAM instead of the strongest reported non-SFT baseline under ideal credibility scores | Llama3-8B · NQ and TriviaQA, 4 high-credibility + 1 low-credibility document | EM: NQ: 29.90 -> 36.90; TriviaQA: 53.50 -> 64.40 | Helped |
| Apply CrAM instead of the strongest reported non-SFT baseline with GPT-generated credibility scores | Qwen1.5-7B · NQ and TriviaQA, 4 high-credibility + 1 low-credibility document | EM: NQ: 21.60 -> 23.10; TriviaQA: 49.50 -> 52.10 | Helped |
| Apply CrAM instead of the strongest reported non-SFT baseline with GPT-generated credibility scores | Llama2-13B · NQ and TriviaQA, 4 high-credibility + 1 low-credibility document | EM: NQ: 23.70 -> 25.10; TriviaQA: 54.40 -> 56.20 | Helped |
| Apply CrAM instead of the strongest reported non-SFT baseline with GPT-generated credibility scores | Llama3-8B · NQ and TriviaQA, 4 high-credibility + 1 low-credibility document | EM: NQ: 26.60 -> 30.70; TriviaQA: 57.70 -> 62.20 | Helped |
| Select influential attention heads rather than modifying every attention head (ablation) | CrAM · NQ and TriviaQA, ideal credibility scores | EM: Qwen1.5-7B: 27.20 -> 29.10 (NQ), 50.60 -> 52.90 (TriviaQA); Llama2-13B: 29.50 -> 33.60, 59.50 -> 59.90; Llama3-8B: 22.40 | Helped |
| Add attention-weight modification based on credibility scores (ablation) | Naive RAG · NQ and TriviaQA, ideal credibility scores | EM: Qwen1.5-7B: 10.50 -> 29.10 (NQ), 25.00 -> 52.90 (TriviaQA); Llama2-13B: 11.90 -> 33.60, 28.00 -> 59.90; Llama3-8B: 16.00 | Helped |
| Increase the number of selected attention heads toward either zero or the maximum number (ablation) | CrAM · NQ, ideal setting | EM | Hurt |
| Vary the dataset size used to identify influential heads (ablation) | CrAM · NQ and TriviaQA | EM: maximum difference of 4% | No clear effect |
| Increase the number of low-credibility documents from 1 to 3 (ablation) | CrAM · NQ, Llama3-8B, ideal and GPT settings | performance | Mixed |
| Replace misinformation with filtered misinformation that removes correct-answer text (ablation) | CrAM · NQ and TriviaQA, ideal scores | EM: Qwen1.5-7B: 29.10 -> 25.90 (NQ), 52.90 -> 51.70 (TriviaQA); Llama2-13B: 33.60 -> 29.90, 59.90 -> 57.90; Llama3-8B: 36.90 | Mixed |
Surprising findings
- CrAM sometimes outperforms Naive RAG with clean four documents under ideal setting, which the authors attribute to misinformation that also denies correct answers; with filtered misinformation this advantage largely disappears.
- The IE distribution across attention heads is roughly normal centered at zero, and performance drops sharply only when selecting near zero or all heads.
Limitations
- Requires a small set of misinformation-polluted examples with incorrect answers to estimate head influence.
- Gains partly depend on misinformation that contains correct answers that CrAM can reuse; with filtered misinformation CrAM is often below Naive RAG.
- Evaluated only on two QA datasets and three open models, with credibility scores either ideal or GPT-generated.
Experimental setup
Datasets: Natural Questions (NQ) TriviaQA Wikipedia dump (Dec 30, 2018)
Benchmarks: Natural Questions (NQ) TriviaQA
Metrics: Exact Match (EM) F1 score accuracy
Baselines: Naive RAG Prompt Based Exclusion CAG-13B Naive LLM
Builds on: causal tracing retrieval-augmented generation CAG
Closest papers by meaning
- Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals (NeurIPS 2025 · similarity 0.87)
- Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting (ICLR 2025 · similarity 0.87)
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (ICLR 2024 · similarity 0.86)
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.86)
- ClashEval: Quantifying the tug-of-war between an LLM’s internal prior and external evidence (NeurIPS 2024 · similarity 0.86)
- Long-form factuality in large language models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.