Research map / Large language models / Retrieval and knowledge
Enhancing Elusive Clues in Knowledge Learning by Contrasting Attention of Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes contrasting attention weights of large and small LMs of the same family to find elusive clues, then uses these to guide token-dropout data augmentation for continual pretraining, improving factual knowledge memorization.
Problem
Causal LMs learn factual knowledge inefficiently from small, knowledge-dense corpora due to long-range dependencies and overfitting to co-occurrence and distracting clues.
Approach
Larger models attend more to non-obvious but important clues (the correct head entity); the attention difference between a large and a small model of the same family identifies these clues, which then guide a token-dropout augmentation that drops obvious tokens more often so elusive clues are preserved.
Key results
- On the biography dataset, attention-difference token-dropout raises Gemma 2 9B company EM from 0.63 (plain finetuning) to 0.90 and LLaMA 3 70B company EM from 0.66 to 0.96.
- On Paragraph-Level Wikipedia QA, the method improves EM over plain finetuning for all four models, e.g., Gemma 2 2B 0.126 to 0.156 and LLaMA 3 8B 0.146 to 0.172.
- Random token-dropout and original-attention dropout give smaller or inconsistent gains compared with attention-difference dropout.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use attention-difference-guided token dropout instead of the strongest baseline | Gemma 2 2B · Biography University | EM: 0.19 -> 0.25 | Helped |
| Use attention-difference-guided token dropout instead of the strongest baseline | Gemma 2 9B · Biography University | EM: 0.61 -> 0.84 | Helped |
| Use attention-difference-guided token dropout instead of the strongest baseline | LLaMA 3 8B · Biography University | EM: 0.30 -> 0.29 | Hurt |
| Use attention-difference-guided token dropout instead of the strongest baseline | LLaMA 3 70B · Biography University | EM: 0.71 -> 0.90 | Helped |
| Use attention-difference-guided token dropout instead of the strongest baseline | Gemma 2 2B · Biography Company | EM: 0.23 -> 0.32 | Helped |
| Use attention-difference-guided token dropout instead of the strongest baseline | Gemma 2 9B · Biography Company | EM: 0.63 -> 0.90 | Helped |
| Use attention-difference-guided token dropout instead of the strongest baseline | LLaMA 3 8B · Biography Company | EM: 0.24 -> 0.42 | Helped |
| Use attention-difference-guided token dropout instead of the strongest baseline | LLaMA 3 70B · Biography Company | EM: 0.71 -> 0.96 | Helped |
| Use attention-difference-guided token dropout instead of the strongest baseline | Gemma 2 2B · Wikipedia QA | EM: 0.145 -> 0.156 | Helped |
| Use attention-difference-guided token dropout instead of the strongest baseline | Gemma 2 9B · Wikipedia QA | EM: 0.205 -> 0.231 | Helped |
Surprising findings
- Random token-dropout hurts university-field EM for Gemma 2 2B and LLaMA 3 8B relative to plain finetuning in the biography task.
- Larger models attend only slightly more to the correct head entity in absolute terms, though the contrast highlights it clearly.
Limitations
- Evaluated only on small synthetic biography data and one Wikipedia QA dataset.
- Requires access to both a large and a small model from the same family to compute attention differences.
Experimental setup
Datasets: synthetic biography dataset Paragraph-Level Wikipedia Question-Answering dataset
Benchmarks: synthetic biography QA Paragraph-Level Wikipedia QA
Metrics: exact match F1
Baselines: Plain finetuning Random token-dropout Token-dropout by attention
Builds on: LoRA RoPE token-dropout data augmentation
Closest papers by meaning
- How new data permeates LLM knowledge and how to dilute it (ICLR 2025 · similarity 0.89)
- Physics of Language Models: Part 3.1, Knowledge Storage and Extraction (ICML 2024 · similarity 0.88)
- Co-occurrence is not Factual Association in Language Models (NeurIPS 2024 · similarity 0.88)
- Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition (ICLR 2025 · similarity 0.88)
- Revealing and Mitigating Over-Attention in Knowledge Editing (ICLR 2025 · similarity 0.87)
- Physics of Language Models: Part 3.2, Knowledge Manipulation (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.