Research map / Large language models / Language, safety and interpretability
Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows attention sinks appear in later tokens (not just the initial token) and that reducing them in selected heads improves LLM accuracy, proposing ACT, a training-free attention calibration method.
Problem
Attention sinks have only been studied at the initial token; it is unclear whether sinks exist elsewhere in the input and whether preserving them helps or hurts LLM accuracy, and whether accuracy can be improved by calibrating attention without weight updates.
Approach
Identifies attention sinks at later tokens (e.g., '.', ':', newline) in intermediate layers, finds that reducing them in selected heads can improve accuracy, and proposes ACT, which filters heads offline on a small held-out set and then, at inference, scales down sink attention and redistributes the mass across other tokens while keeping rows summing to one.
Key results
- ACT improves Llama-30B zero-shot average accuracy on domain-specific multiple-choice tasks by 7.30% (62.08 vs 69.19 type gains reported as up to 7.30% average), with a peak gain of 13.26% on Hellaswag.
- On text classification, ACT raises Llama-30B zero-shot average accuracy by 5.12%, including a 16.16% gain on AGNews.
- ACT improves MMLU accuracy for Llama2-7B-chat, GPT-J, Vicuna-7B, and OPT-2.7B by 0.32% to 1.09%, and raises MT-Bench score by 0.088 to 0.134 for Llama2 chat models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply ACT to calibrate attention sinks during inference | Llama-30B · Domain-specific multiple choice, 0-shot | accuracy (Avg.): 61.89 -> 69.19; 7.30 | Helped |
| Apply ACT to calibrate attention sinks during inference | Llama2-7B-chat · Text classification, 0-shot | accuracy (Avg.): 65.07 -> 66.36; 1.29 | Helped |
| Apply ACT to calibrate attention sinks during inference | Llama2-7B-chat · MT-Bench | score: 6.272 -> 6.406; 0.134 | Helped |
| Reduce attention at middle-input sinks and redistribute it to other tokens rather than using temperature scaling (ablation) | Llama2-7B-chat · MMLU | accuracy: 44.89 -> 46.82 | Helped |
| Reverse the calibration by increasing sink attention at the expense of other tokens (ablation) | Llama2-7B-chat · MMLU | accuracy: 46.82 -> 46.21 | Hurt |
| Redistribute recovered attention only to question tokens instead of across all tokens (ablation) | Llama2-7B-chat · MMLU | accuracy: 46.82 -> 46.10 | Hurt |
| Redistribute recovered attention only to choice tokens instead of across all tokens (ablation) | Llama2-7B-chat · MMLU | accuracy: 46.82 -> 45.24 | Hurt |
| Vary attention-sink threshold α among 3, 5, and 7 (ablation) | Llama2-7B-chat · Text classification datasets | accuracy | No clear effect |
| Vary sink attention reduction factor β from 0.7 to 0.1 (ablation) | Llama2-7B-chat · MMLU | accuracy: 46.77; 46.81; 46.82; 46.79; 46.65 | No clear effect |
| Increase the number of head-filtering samples M (ablation) | Llama2-7B-chat · MMLU | accuracy: 46.50 -> 46.56 -> 46.82 -> 46.91 | Helped |
| Calibrate only a subset rather than all selected attention heads (ablation) | Llama2-7B-chat · SST2, AGNews, PIQA, ARCC | accuracy (Avg.): 71.57 -> 73.34 -> 72.88 -> 73.44 -> 73.84 | Helped |
| Reduce attention sinks in individual heads (ablation) | Llama2-7B-chat · MMLU | accuracy | Mixed |
Surprising findings
- Reducing attention sinks in later tokens improves accuracy in more than 76.8% of cases, contrary to StreamingLLM's advice to preserve sinks.
- Calibrating some heads reduces accuracy by up to 0.39%, so head selection is necessary.
Limitations
- Gains are modest for some models and settings (e.g., below 1% on MMLU and several few-shot settings).
- Head filtering requires an offline held-out evaluation per task, so it is not fully task-agnostic.
- Calibrating some heads can hurt accuracy, and the mechanism behind sink-driven accuracy changes is only hypothesized.
Experimental setup
Datasets: Hellaswag ARC-E PIQA OpenBookQA ARC-C COPA CommonsenseQA MMLU SST2 SST5 MR AGNews
Benchmarks: Hellaswag ARC-E PIQA OpenBookQA ARC-C COPA CommonsenseQA MMLU SST2 SST5 MR AGNews
Metrics: accuracy F1 exact match MT-Bench score
Baselines: Vanilla inference Temp (temperature scaling) Inv-temp Inv-ours in-context learning (1/3/5-shot)
Builds on: StreamingLLM attention sink
Closest papers by meaning
- When Attention Sink Emerges in Language Models: An Empirical View (ICLR 2025 · similarity 0.94)
- On the Nature of Attention Sink that Shapes Decoding Strategy in Omni-LLMs (NeurIPS 2026 · similarity 0.92)
- Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings (NeurIPS 2025 · similarity 0.91)
- See What You Are Told: Visual Attention Sink in Large Multimodal Models (ICLR 2025 · similarity 0.90)
- Efficient Streaming Language Models with Attention Sinks (ICLR 2024 · similarity 0.90)
- Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.