Research map / Large language models / Retrieval and knowledge
Controllable Context Sensitivity and the Knob Behind It
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that fine-tuned instruct LLMs can be steered between context and prior knowledge via a single 1-dimensional subspace in one layer, and that this subspace transfers to non-fine-tuned and base models of the same family.
Problem
Language models must trade off reliance on in-context information versus parametric prior knowledge, and it is unclear whether this choice is controllable and what internal mechanism implements it.
Approach
Defines a controllable context sensitivity (CCS) task with explicit intent instructions, uses a linear-time activation-patching search to locate relevant layers, and learns a rank-1 orthogonal projection at one layer whose scalar value acts as a knob between following context and prior knowledge.
Key results
- Fine-tuned Llama-3.1-8B-Instruct reaches about 90% pair accuracy in-domain on CCS-BaseFake (ICL about 88%), with degradation on MHFake and Arithmetic.
- Steering a single 1-dimensional subspace in layer 16 achieves 83% PairAcc on the fine-tuned model versus a 95% instruction baseline, and 73% zero-shot on the non-fine-tuned instruct model.
- Correlation of 0.908 between model performance and the separation of subspace values across intents.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Steer the rank-1 context-versus-prior subspace | Llama-3.1-8B-Instruct FT · CCSbasefake | PairAcc: 95% -> 83% | Hurt |
| Steer the rank-1 context-versus-prior subspace | Llama-3.1-8B-Instruct ZS · CCSbasefake | PairAcc | Helped |
| Steer the rank-1 context-versus-prior subspace | Llama-3.1-8B base ZS · CCSbasefake | PairAcc | No clear effect |
| Use 10-shot in-context learning instead of fine-tuning | Llama-3.1-8B-Instruct · CCSbasefake in-domain test | PairAcc: ≈ 90% vs. ≈ 88% | No clear effect |
| Patch MLP outputs across all layers rather than MHA outputs of 7 layers (ablation) | Llama-3.1-8B-Instruct FT · CCSbasefake, prior-answer patching | SRC PRI probability: 86% vs. 54% | Hurt |
| Omit patching layer 24 (ablation) | Llama-3.1-8B-Instruct FT context-answer patching · Context-answer patching | alternate context answer becomes top-probability answer | Hurt |
| Patch only layers 15-16 (ablation) | Llama-3.1-8B-Instruct FT prior-answer patching · Prior-answer patching | SRC PRI probability | Hurt |
| Remove layer 24 from prior-answer patching (ablation) | Llama-3.1-8B-Instruct FT · Prior-answer patching | SRC PRI probability | Hurt |
| Train on the instruction intent format and evaluate with the context-weight format | Llama-3.1-8B-Instruct · CCSbasefake intent-format generalization | PairAcc | Hurt |
| Evaluate on arithmetic rather than the training-domain task | Llama-3.1-8B-Instruct FT and ICL · CCSarithmetic | PairAcc | Hurt |
Surprising findings
- The same subspace learned on a fine-tuned model steers non-fine-tuned instruct models zero-shot, and base models under ICL.
- Patching all MLP outputs at the last token does not yield the prior answer, so MLPs at the last position contribute little direct answer computation.
Limitations
- Models are better at context-agreeing answers when the answer is explicitly stated in context; reasoning-based use of context is less well covered.
- Subspace intervention is ineffective on the fine-tuned base model.
- Generalization to out-of-domain datasets such as arithmetic is moderate, and Mistral generalizes worse.
Experimental setup
Datasets: CCS-BaseFake CCS-MHFake CCS-Arithmetic BaseFakepedia MHFake Arithmetic
Benchmarks: CCS-BaseFake CCS-MHFake CCS-Arithmetic
Metrics: pair accuracy PairAcc under subspace steering
Baselines: zero-shot prompting with intent instruction in-context learning instruct model without fine-tuning
Builds on: distributed alignment search activation patching Token Identity Patchscope linear subspace hypothesis Fakepedia
Closest papers by meaning
- Making Retrieval-Augmented Language Models Robust to Irrelevant Context (ICLR 2024 · similarity 0.86)
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.86)
- A Controllable Examination for Long-Context Language Models (NeurIPS 2025 · similarity 0.85)
- TAIA: Large Language Models are Out-of-Distribution Data Learners (NeurIPS 2024 · similarity 0.84)
- The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation (NeurIPS 2025 · similarity 0.84)
- PICASO: Permutation-Invariant Context Composition with State Space Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.