Research map / Large language models / Language, safety and interpretability
Expert Heads: Robust Evidence Identification for Large Language Models
Qi Wu, Jianfeng Qu, Ximing Li, Zhixu Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Response-as-Source instead of Question-as-Source Expert Heads | LLaMA-3-8B-Instruct · HotpotQA | Precision@2: 88.23 -> 90.72 | Improved |
| use Response-as-Source instead of Question-as-Source Expert Heads | Qwen-2.5-7B-Instruct · 2WikiMultiHopQA | Precision@2: 74.69 -> 77.94 | Improved |
| use Response-as-Source instead of Question-as-Source Expert Heads | Mistral-7B-Instruct-v0.3 · 2WikiMultiHopQA | Precision@2: 70.72 -> 73.27 | Improved |
| use Expert Head voting instead of direct LLM Rank | Mistral-7B-Instruct-v0.3 · HotpotQA | Precision@2: 36.45 -> 88.89 | Improved |
| use Expert Head voting instead of direct LLM Rank | LLaMA-3-8B-Instruct · HotpotQA | Precision@2: 66.31 -> 90.72 | Improved |
| use Expert Head voting instead of direct LLM Rank | LLaMA-3-8B-Instruct · 2WikiMultiHopQA | Precision@2: 76.49 -> 73.47 | Mixed |
| use Expert Head voting instead of direct LLM Rank | Qwen-2.5-7B-Instruct · 2WikiMultiHopQA | Precision@2: 76.57 -> 77.94 | No clear effect |
| use LLaMA Expert Heads (R) instead of the best dense retriever BGE | LLaMA-3-8B-Instruct · HotpotQA | Precision@2: 75.23 -> 90.72 | Improved |
| use LLaMA Expert Heads (R) instead of BGE | LLaMA-3-8B-Instruct · 2WikiMultiHopQA | Precision@2: 77.12 -> 77.30 | No clear effect |
| use Mistral Expert Heads (Q) instead of BGE | Mistral-7B-Instruct-v0.3 · 2WikiMultiHopQA | Precision@2: 77.12 -> 70.72 | Hurt |
| use all heads of a single middle layer as expert heads vs other layers (ablation) | LLaMA-3-8B-Instruct · HotpotQA | Precision@2 | Mixed |
| use stricter activation thresholds for Response-as-Source Expert Heads (ablation) | LLaMA-3-8B-Instruct · HotpotQA | Precision@2 | Improved |
Closest papers by meaning
- Learning Evidence Highlighting for Frozen LLMs (NeurIPS 2026 · similarity 0.88)
- Retrieval Head Mechanistically Explains Long-Context Factuality (ICLR 2025 · similarity 0.87)
- Listening to the Wise Few: Query–Key Alignment Unlocks Latent Correct Answers in Large Language Models (NeurIPS 2026 · similarity 0.87)
- Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding (CVPR 2026 · similarity 0.87)
- On the Role of Attention Heads in Large Language Model Safety (ICLR 2025 · similarity 0.86)
- Logit-Contribution Scoring Identifies Non-Literal Retrieval Heads (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
