Research map / Large language models / Language, safety and interpretability
Function Vectors in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies function vectors: sums of mean outputs of a small set of causal attention heads that transport a compact task representation in autoregressive LMs, which can be added to a residual stream to trigger ICL tasks in zero-shot and natural-text contexts.
Problem
How in-context learning tasks are represented and executed inside autoregressive transformer LMs; whether task representations can be causally extracted as compact vectors that transfer beyond the original ICL context.
Approach
Use causal mediation analysis (activation patching of mean attention-head outputs over corrupted prompts) to find a small set of causal attention heads, and sum their task-conditioned mean outputs into a single function vector that is added to a hidden layer to trigger the task in new contexts.
Key results
- Adding the function vector to GPT-J raises zero-shot antonym-style accuracy to 57.5% versus 5.5% for the base model, and shuffled-label accuracy to 90.8% versus 39.1%.
- On Llama 2 (70B), the FV reaches 83.8% zero-shot accuracy versus 8.2% baseline across the 6 tasks.
- Reconstructions matching only the top 100 decoded vocabulary tokens of an FV perform poorly as function vectors (e.g., Antonym 4.8% vs 48.2%), suggesting the FV carries information beyond its vocabulary.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a function vector to shuffled-label inputs | GPT-J · 6 representative tasks, shuffled-label | accuracy: 39.1 ± 1.2% -> 90.8±0.9% | Helped |
| Add a function vector to zero-shot inputs | GPT-J · 6 representative tasks, zero-shot | accuracy: 5.5± 0.8% -> 57.5 ± 1.7% | Helped |
| Use the causal-mediation function vector instead of a layer-average vector | GPT-J · 6 representative tasks, shuffled-label and zero-shot | accuracy: 79.5±3.1% -> 90.8±0.9%; 9.5±1.8% -> 57.5 ± 1.7% | Helped |
| Add a function vector to zero-shot inputs | GPT-NeoX · 6 representative tasks, zero-shot | accuracy: 6.7±0.1% -> 57.1±1.5% | Helped |
| Add a function vector to zero-shot inputs | Llama 2 (70B) · 6 representative tasks, zero-shot | accuracy: 8.2±0.7% -> 83.8±0.7% | Helped |
| Reconstruct the function vector from only the top 100 decoded tokens instead of the original vector (ablation) | GPT-J function vectors · Zero-shot; six tasks | accuracy: Antonym 48.2 ± 2.0 % -> 4.8 ± 2.0%; Capitalize 70.5 ± 2.4% -> 5.7 ± 2.2%; Country-Capital 83.2 ± 2.7% -> 58.1 ± 18.5%; E | Hurt |
| Reconstruct the function vector from all decoded tokens instead of the original vector (ablation) | GPT-J function vectors · Zero-shot; six tasks | accuracy: Antonym 48.2 ± 2.0 % -> 39.6 ± 2.6%; Capitalize 70.5 ± 2.4% -> 51.5 ± 11.6%; Country-Capital 83.2 ± 2.7% -> 29.0 ± 15.1% | Mixed |
| Compose task vectors as vBD* = vAD + vBC - vAC (ablation) | GPT-J function vectors · List-oriented tasks, zero-shot | accuracy: Last-Antonym 0.02 ± 0.01 -> 0.07 ± 0.02; Last-Capitalize 0.64 ± 0.03 -> 0.76 ± 0.04; Last-Country-Capital 0.15 ± 0.03 -> | Mixed |
Surprising findings
- Function vectors built from a small set of attention heads can induce cyclic tasks such as antonym, which cannot be simple constant word-embedding offsets.
- Reconstructed vectors that decode to the same top vocabulary as the FV recover little of its performance, indicating vocabulary alone does not explain FV function.
Limitations
- Tasks are relatively simple, mostly single-token word-level functions.
- FVs are not a complete account of ICL; the mechanism only explains one level of mediation.
- Performance is often low for some tasks (e.g., English-French zero-shot on GPT-NeoX and Llama 2 70B) and the effect drops sharply in late layers.
Experimental setup
Datasets: antonym and synonym word pairs (adapted from prior lexical data) English-French, English-German, English-Spanish translation pairs SST-2 sentiment subset CommonsenseQA AG News ChatGPT-generated country-capital and capitalize-first-letter lists National Parks (Wikipedia)
Benchmarks: antonym capitalize country-capital english-french present-past singular-plural 34 additional ICL tasks natural text completions list-composition tasks
Metrics: top-1 accuracy average indirect effect KL divergence
Baselines: GPT-J zero-shot layer-averaged hidden state (h-bar) shuffled-label ICL baseline
Builds on: causal mediation analysis logit lens induction heads task vectors
Closest papers by meaning
- Which Attention Heads Matter for In-Context Learning? (ICML 2025 · similarity 0.88)
- Do different prompting methods yield a common task representation in language models? (NeurIPS 2025 · similarity 0.88)
- Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective (ICML 2025 · similarity 0.88)
- In-Context Learning through the Bayesian Prism (ICLR 2024 · similarity 0.87)
- Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning (NeurIPS 2026 · similarity 0.87)
- In-Context Learning with Representations: Contextual Generalization of Trained Transformers (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.