Research map / Large language models / Reasoning and chain of thought

Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models

ICLR 2025 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that isolated demonstration tokens with low self-aggregation can distract few-shot CoT reasoning, and proposes Few-shot Attention Intervention (FAI), which blocks attention from such tokens to the output position, improving accuracy on several reasoning benchmarks.

Problem

Few-shot chain-of-thought performance is sensitive to demonstrations, and isolated tokens or segments within demonstrations can distract LLMs into incorrect reasoning, with no clear internal mechanism explaining this.

Approach

Tokens that have not aggregated information from other tokens (low self-attention relative to a position-based threshold) and directly feed the prediction position are identified per layer from attention matrices; FAI zeroes their attention to the output token in the next layer, suppressing the distraction while preserving CoT's positive effect.

Key results

• FAI improves Llama3-8B on AQuA from 40.94 to 46.85 (about 5.91 points), the paper's headline gain.
• On the GSM8K robustness analysis, FAI raises Llama3-8B average accuracy across shot settings from 70.61 to 72.22 and Mistral-7B from 37.16 to 38.97.
• FAI intervenes on roughly 15.8% of demonstration tokens for Llama3-8B in the one-shot GSM8K setting.

Measured effects reported in the paper

Surprising findings

• Blocking all demonstration-to-prediction attention reduces distraction but substantially hurts accuracy on correctly answered samples, while the targeted FAI intervention does not.

Limitations

• Experiments limited to models of 7B to 13B parameters, plus Llama-3-70B in a subset of tables.
• Error-type attribution relies on manual inspection of 180 samples, and the 60% distraction estimate is approximate.
• Some datasets (e.g., GPT2-XL, GPT-NEO) show very low GSM8K accuracy, limiting conclusions there.

Experimental setup

Datasets: GSM8K AQuA CSQA BIG-Bench Hard date understanding BIG-Bench Hard sports understanding last letter concatenation

Benchmarks: AQuA GSM8K CSQA Date Understanding Sport Understanding Last letter concatenation

Metrics: accuracy rate of answer following rationale (RAFR)

Baselines: GPT2-XL GPT-NEO Llama-3-8B-Instruct Llama-3-70B-Instruct Llama-2-13B-Chat Mistral-7B-v0.2

Builds on: chain-of-thought prompting attention sink saliency-based attribution

Closest papers by meaning

• From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency (ICLR 2025 · similarity 0.90)
• Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns (AAAI 2025 · similarity 0.90)
• Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration (ICML 2024 · similarity 0.90)
• Attention Illuminates LLM Reasoning: The Uncovered Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization (ICML 2026 · similarity 0.89)
• SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought (NeurIPS 2025 · similarity 0.89)
• Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Models (NeurIPS 2025 · similarity 0.89)