Research map / Large language models / Language, safety and interpretability
Improving Instruction-Following in Language Models through Activation Steering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives instruction-specific steering vectors via difference-in-means on residual-stream activations with and without instructions, and uses them to steer LLMs toward format, length, and word-inclusion/exclusion constraints on IFEval.
Problem
Language models often fail to follow fine-grained, verifiable instructions (format, length, keyword constraints) and drift from them during generation; it is unclear whether such instructions are linearly represented and can be used to control behavior at inference time.
Approach
Compute a steering direction per instruction as the mean difference in last-token residual-stream activations between base queries with and without the instruction, then add a scaled version at a single layer during generation; extend this to multiple instructions, word exclusion via subtraction, and cross-model transfer from instruction-tuned to base models.
Key results
- Steering raises instruction-following accuracy without text instructions from about 10% to about 30% across models on format constraints.
- Steering for length (conciseness, weight c=20) significantly improves compliance with maximum sentence-count constraints for Phi-3 across n=1 to 5.
- On Gemma 2 2B, cross-model steering (vectors from instruction-tuned model applied to base model) outperforms same-model steering in both settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply format-instruction steering without explicit text instructions | Phi-3, Gemma 2 2B, Gemma 2 9B, and Mistral 7B · IFEval format instructions | instruction-following accuracy: around 10% -> approximately 30% | Helped |
| Apply format-instruction steering when instructions are explicitly present | Phi-3, Gemma 2 2B, Gemma 2 9B, and Mistral 7B · IFEval format instructions | instruction-following accuracy | Helped |
| Steer for conciseness with increasing weight | Phi-3 · IFEval base prompts without explicit length instructions | response length: 15 -> 9 -> 10 -> 3 -> 1 sentences; 324 -> 228 -> 203 -> 81 -> 23 words | Helped |
| Apply conciseness steering with c=20 | Phi-3 · IFEval maximum sentence constraints, n in {1, ..., 5} | length-constraint adherence | Helped |
| Apply word-inclusion steering | Phi-3 and other evaluated models · IFEval keyword inclusion | keyword inclusion | Helped |
| Use inclusion vectors and subtract them to steer away from a keyword, rather than using exclusion vectors | Phi-3 and other evaluated models · IFEval keyword exclusion | keyword exclusion | Helped |
| Steer simultaneously for two instructions | Phi-3 · IFEval format and length; lowercase and keyword exclusion | instruction adherence | Helped |
| Transfer steering vectors from instruction-tuned Gemma 2 2B to the base model instead of computing vectors on the base model | Gemma 2 2B base · IFEval format instructions, with and without explicit instructions | instruction-following performance | Helped |
| Apply instruction-tuned Gemma 2 9B conciseness vectors to the base model | Gemma 2 9B base · Length control | response length | Helped |
| Apply format steering | Four evaluated models · Response quality on base queries | response quality score | Hurt |
| Use averaged steering vectors for variable, input-dependent End Checker text | Phi-3 · IFEval End Checker format instruction | instruction-following accuracy | Hurt |
| Apply steering to Constrained Response or Title instructions | Evaluated models · IFEval Constrained Response and Title instructions | instruction-following accuracy | Hurt |
Surprising findings
- Adding word-exclusion steering vectors directly increased the probability of the excluded words, so exclusion was implemented by subtracting inclusion vectors.
- Steering vectors computed on instruction-tuned models can outperform same-model steering when applied to base models (Gemma 2 2B).
Limitations
- Fixed steering weight across generation; instruction drift handled only by single-layer intervention.
- Perplexity via GPT-2 may miss factual or logical degradation; some steering causes nonsensical output or repetition.
- Evaluation limited to a few instruction types and four model families; cross-architecture transfer not explored.
Experimental setup
Datasets: IFEval synthetically augmented IFEval prompts (GPT-4o generated)
Benchmarks: IFEval
Metrics: loose instruction-following accuracy word/sentence count compliance keyword presence accuracy response quality score (GPT-4o yes/no questions) perplexity (GPT-2)
Baselines: no steering (same model) adding instruction as text
Builds on: IFEval difference-in-means steering activation addition
Closest papers by meaning
- Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior (NeurIPS 2026 · similarity 0.90)
- Steering When Necessary: Flexible Steering Large Language Models with Backtracking (NeurIPS 2025 · similarity 0.90)
- Instruction Tuning With Loss Over Instructions (NeurIPS 2024 · similarity 0.90)
- Evaluating the Zero-shot Robustness of Instruction-tuned Language Models (ICLR 2024 · similarity 0.89)
- Iterative Vectors: In-Context Gradient Steering without Backpropagation (ICML 2025 · similarity 0.89)
- Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.