Research map / Large language models / Language, safety and interpretability
Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PASTA, a post-hoc inference-time method that reweights attention scores on a profiled subset of heads so LLMs attend to user-emphasized spans, without changing weights.
Problem
LLMs struggle to focus on user-specified information (instructions, new facts in context, long contexts) and cannot interpret text emphasis markers like bold or italics, which are rarely seen in pretraining.
Approach
Identify a small set of attention heads via multi-task profiling (intersection of top-k heads across tasks) and downscale attention to non-emphasized tokens by a factor alpha (renormalized), steering the model toward user-highlighted spans at inference time without weight updates or gradients.
Key results
- PASTA (multi-task) on LLaMA-7B reaches 95.46 average across the four tasks versus 73.45 for few-shot prompting and 67.29 for zero-shot, an average gain of about 22 points.
- On GPT-J, PASTA (multi-task) achieves 94.96% BiasBios accuracy, 16.32 points above the best baseline.
- On LLaMA-7B JSON Formatting, PASTA reaches 96.64% format accuracy and 85.09% prediction accuracy, exceeding few-shot prompting by about 11 points on prediction accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply multi-task PASTA instead of few-shot prompting | LLaMA-7B · JSON Formatting | Format accuracy: 84.85 -> 96.64 | Helped |
| Apply multi-task PASTA instead of zero-shot prompting | LLaMA-7B · Pronouns Changing | Accuracy: 71.84 -> 96.42 | Helped |
| Apply multi-task PASTA instead of few-shot prompting | LLaMA-7B · BiasBios | Accuracy: 88.79 -> 95.28 | Helped |
| Apply multi-task PASTA instead of few-shot prompting | LLaMA-7B · CounterFact | Efficacy score: 87.45 -> 99.60 | Helped |
| Apply multi-task PASTA instead of few-shot prompting | GPT-J-6B · JSON Formatting | Format accuracy: 84.15 -> 91.50 | Helped |
| Apply multi-task PASTA instead of zero-shot prompting | GPT-J-6B · Pronouns Changing | Accuracy: 39.88 -> 92.96 | Helped |
| Apply multi-task PASTA instead of marked prompting | GPT-J-6B · BiasBios | Accuracy: 78.64 -> 94.96 | Helped |
| Apply task-agnostic PASTA instead of few-shot prompting | GPT-J-6B · CounterFact | Efficacy score: 68.34 -> 99.54 | Helped |
| Select task-specific heads instead of intersection-selected heads (ablation) | PASTA on LLaMA-7B · JSON Formatting | Format accuracy: 96.64 -> 95.56 | No clear effect |
| Select union heads instead of intersection-selected heads (ablation) | PASTA on LLaMA-7B · JSON Formatting | Format accuracy: 96.64 -> 88.42 | Hurt |
| Select task-specific heads instead of intersection-selected heads (ablation) | PASTA on GPT-J-6B · JSON Formatting | Format accuracy: 91.50 -> 85.71 | Mixed |
| Select union heads instead of intersection-selected heads (ablation) | PASTA on GPT-J-6B · JSON Formatting | Format accuracy: 91.50 -> 72.61 | Hurt |
Surprising findings
- Steering all attention heads degrades performance below zero-shot, indicating that head selection is necessary.
- Marked prompting with asterisks or quotes often performs worse than zero-shot, suggesting LLMs do not interpret emphasis markers well.
Limitations
- Head selection requires labeled profiling samples from multiple tasks, and multi-task performance benefits from labeled data for the evaluated task.
- Higher numbers of steered heads trade off generation quality (fluency, prediction accuracy) against instruction adherence.
- Evaluated only on a few tasks and models up to 13B parameters, with results on the task-agnostic setting weaker for some metrics.
Experimental setup
Datasets: BiasBios CounterFact JSON Formatting (constructed from BiasBios) Pronouns Changing (constructed from BiasBios)
Benchmarks: JSON Formatting Pronouns Changing BiasBios CounterFact
Metrics: format accuracy prediction accuracy accuracy all-changed accuracy efficacy score paraphrase score fluency (bi-gram/tri-gram entropy)
Baselines: Zero-shot prompting Marked prompting Few-shot prompting
Builds on: few-shot prompting chain-of-thought prompting
Closest papers by meaning
- Focus On This, Not That! Steering LLMs with Adaptive Feature Specification (ICML 2025 · similarity 0.86)
- Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration (ICML 2024 · similarity 0.86)
- Leveraging Attention to Effectively Compress Prompts for Long-Context LLMs (AAAI 2025 · similarity 0.85)
- Evaluating Large Language Models at Evaluating Instruction Following (ICLR 2024 · similarity 0.85)
- Prompt Highlighter: Interactive Control for Multi-Modal LLMs (CVPR 2024 · similarity 0.85)
- Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.