Research map / Large language models / Instruction and fine-tuning
Focus On This, Not That! Steering LLMs with Adaptive Feature Specification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Focus Instruction Tuning (FIT), which fine-tunes LLMs with natural-language focus/ignore instructions so the same input yields different outputs depending on which features are specified, evaluated on synthetic and real spurious-correlation NLP tasks.
Problem
Instruction-tuned LLMs rely on spurious or biased features from training data and become misaligned, while existing inference-time steering methods are post-hoc and not intrinsic to the model.
Approach
Augment instruction tuning with focus instructions (focus on, ignore, or both) and train on focus labels that equal the ground truth or the spurious label depending on the instruction, so the model learns to condition its prediction on user-specified features in natural language.
Key results
- On SMNLI, FIT achieves very high focus accuracy on core and genre features, significantly beating baselines; FIT keeps over 80% focus accuracy on unseen genres under shift.
- On BBQ, FIT gives high and comparable focus accuracy on both seen and held-out bias features.
- Instruction-following rating changes are at most 0.05 points (Llama 3.51->3.45, Vicuna 3.46->3.50), none significant (p>0.05); MMLU accuracy changes by at most 0.8%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Focus Instruction Tuning | Llama-3.1-8B-Instruct · Alpaca-GPT | mean rating: 3.51 -> 3.45 | No clear effect |
| Apply Focus Instruction Tuning | Mistral-7B-Instruct-v0.3 · Alpaca-GPT | mean rating: 3.65 -> 3.65 | No clear effect |
| Apply Focus Instruction Tuning | Vicuna-13B-v1.5 · Alpaca-GPT | mean rating: 3.46 -> 3.50 | No clear effect |
| Apply Focus Instruction Tuning | Llama-3.1-8B-Instruct · MMLU zero-shot | accuracy: 30.4 -> 29.6 | No clear effect |
| Apply Focus Instruction Tuning | Mistral-7B-Instruct-v0.3 · MMLU zero-shot | accuracy: 29.4 -> 29.0 | No clear effect |
| Apply Focus Instruction Tuning | Llama-3.1-8B-Instruct · MMLU zero-shot | perplexity: 6.29 -> 2.79 | Helped |
| Apply Focus Instruction Tuning | Mistral-7B-Instruct-v0.3 · MMLU zero-shot | perplexity: 15.2 -> 5.22 | Helped |
Surprising findings
- Latent steering methods were reported to significantly underperform SFT methods in prior findings, so the authors compare against SFT rather than latent steering baselines.
Limitations
- Requires annotated spurious features and focus labels, though the authors argue this is not strictly limiting.
- Experiments focus on classification and MCQA; NLG extension is only an initial BBQ-NLG ablation.
- Low focus accuracy when focusing on spurious features for unseen genres, since their spurious labels are not learned in training.
Experimental setup
Datasets: MNLI SST-5 BBQ Alpaca-GPT MMLU
Benchmarks: SMNLI SS BBQ BBQ-NLG MMLU Alpaca-GPT
Metrics: focus accuracy perplexity accuracy average instruction-following rating Wilcoxon signed-rank p-value
Baselines: few-shot prompting SFT(y_focus) zero-shot vanilla SFT(y)
Builds on: instruction tuning conditional SFT LoRA MNLI BBQ
Closest papers by meaning
- Model Whisper: Steering Vectors Unlock Large Language Models’ Potential in Test-Time (AAAI 2026 · similarity 0.89)
- Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior (NeurIPS 2026 · similarity 0.89)
- Steering When Necessary: Flexible Steering Large Language Models with Backtracking (NeurIPS 2025 · similarity 0.89)
- Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectors (ICLR 2025 · similarity 0.89)
- LensLLM: Unveiling Fine-Tuning Dynamics for LLM Selection (ICML 2025 · similarity 0.89)
- A Closer Look at the Limitations of Instruction Tuning (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.