Research map / Large language models / Language, safety and interpretability
Do different prompting methods yield a common task representation in language models?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends function vector extraction from demonstrations to textual instructions and shows instruction FVs boost zero-shot accuracy, partly overlap with demonstration FVs in attention heads, and mainly use different heads, with post-training (SFT/DPO) driving instruction FV quality.
Problem
Whether different task presentations (demonstrations vs. textual instructions) of the same task induce a common task representation inside language models, and how such representations can be extracted and compared.
Approach
Generalizes function vector extraction from in-context demonstrations to arbitrary task specifications (LLM-generated instructions) using three uninformative baselines (equiprobable token sequences, real texts, other-task instructions), then compares head overlap, joint intervention effects, and cross-presentation localization to test for a shared task representation.
Key results
- Instruction FVs raise zero-shot accuracy from below 20% to above 50% in the best models.
- Adding demonstration and instruction FVs jointly outperforms either alone in most models, with the exception of base Llama-3.1-8B.
- Demonstration and instruction FVs share few top heads (7 of 20 in Llama-3.2-3B models, 4 in Llama-3.2-1B models).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add an instruction-derived function vector to the residual stream | Language models evaluated zero-shot across the task set · Zero-shot tasks | accuracy | Helped |
| Add both demonstration-derived and instruction-derived function vectors | Language models evaluated with function-vector interventions · Zero-shot and shuffled 10-shot evaluations | accuracy | Helped |
| Use post-trained-model instruction FVs to steer base models | Base language models · Zero-shot tasks | accuracy | Mixed |
| Add an instruction FV extracted from an OLMo-2 model after supervised fine-tuning | OLMo-2-1124-7B · Zero-shot tasks | accuracy | Helped |
| Add an instruction FV extracted from an OLMo-2 model after preference optimization | OLMo-2-1124-7B · Zero-shot tasks | accuracy | Helped |
| Add an instruction FV extracted after the final RL stage | OLMo-2-1124-7B · Zero-shot tasks | accuracy | No clear effect |
| Add the same function vector twice instead of combining demonstration and instruction FVs (ablation) | Language models receiving two FV interventions · Zero-shot and shuffled 10-shot evaluations | accuracy | Mixed |
| Use instruction-identified heads with demonstration activations rather than demonstration-identified heads with instruction activations (ablation) | Incongruent function vectors · Shuffled 10-shot and zero-shot evaluations | accuracy | Mixed |
| Replace the regular function-vector heads with unrelated heads or the lowest-scoring heads (ablation) | Function-vector interventions · Task evaluations | accuracy | Hurt |
| Use post-trained rather than base-model instruction FVs in OLMo-2 (ablation) | OLMo-2-1124-7B instruction FVs · Zero-shot tasks | accuracy | Helped |
| Use post-trained rather than base-model models for demonstration-based ICL (ablation) | OLMo-2-1124-7B demonstration FVs · Demonstration-based ICL | accuracy | No clear effect |
Surprising findings
- Adding the same FV twice performs similarly to adding both FVs for some models, suggesting part of the joint benefit comes from intervention magnitude.
- Instruction FVs from post-trained models transfer to steer base models, nearly recovering post-trained zero-shot accuracy in three of four base models.
Limitations
- Limited to ~50 minimal, simple tasks; classification tasks with output-format requirements were omitted.
- Fixed intervention depth at |L/3| is somewhat arbitrary; results at optimal depth differ.
- Only a few model sizes (1B-8B) are examined, so scaling is not studied.
Experimental setup
Datasets: function vectors task suite (~50 tasks) WikiText-103-v1
Benchmarks: function vectors task suite (~50 tasks)
Metrics: accuracy causal indirect effect SEM
Baselines: demonstration function vectors
Builds on: function vectors in-context learning task vectors indirect effect causal scoring
Closest papers by meaning
- Decomposing how prompting steers behavior (NeurIPS 2026 · similarity 0.88)
- Which Attention Heads Matter for In-Context Learning? (ICML 2025 · similarity 0.88)
- Capturing In-Context Learning Dynamics with Task Operators (NeurIPS 2026 · similarity 0.88)
- Function Vectors in Large Language Models (ICLR 2024 · similarity 0.88)
- Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning (NeurIPS 2026 · similarity 0.88)
- Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.