Research map / Large language models / Language, safety and interpretability
Steering Evaluation-Aware Language Models To Act Like They Are Deployed
Tim Tian Hua, Andrew Qin, Samuel Marks, Neel Nanda
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add activation steering toward deployment (mean contrastive vector, layers 10-30, strength 3.6) | evaluation-aware Llama Nemotron · BigCodeBench, evaluation contexts | type hint rate: 80-86% -> less than 11% | Improved |
| add activation steering toward deployment vs prompted deployment | evaluation-aware Llama Nemotron · BigCodeBench, Wood Labs cue | type hint rate: 34% (prompted deployment) vs 10% (steered) | Improved |
| add activation steering toward evaluation | evaluation-aware Llama Nemotron · BigCodeBench, deployment context | type hint rate: 34% -> over 95% | Improved |
| steer toward deployment (ablation) | evaluation-aware Llama Nemotron · BigCodeBench | accuracy (%): 56.8 -> 55.2 | No clear effect |
| steer toward evaluation with same strength (ablation) | evaluation-aware Llama Nemotron · BigCodeBench | accuracy (%): 56.8 -> 43.1 | Hurt |
| steer with 24 random Gaussian vectors instead of contrastive vector (ablation) | evaluation-aware Llama Nemotron · BigCodeBench, evaluation contexts | type hint rate | No clear effect |
| prepend all 16 contrastive deployment prompts instead of steering (ablation) | evaluation-aware Llama Nemotron · BigCodeBench, Wood Labs cue | type hint rate | Mixed |
| remove situational awareness hint from system prompt (ablation) | evaluation-aware Llama Nemotron · BigCodeBench | type hint gap eval vs deployment | Mixed |
Closest papers by meaning
- Routing Subspaces: Auditing Evaluation-to-Deployment Mismatch in Fine-Tuned Language Models (NeurIPS 2026 · similarity 0.87)
- Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models (NeurIPS 2026 · similarity 0.87)
- Steering When Necessary: Flexible Steering Large Language Models with Backtracking (NeurIPS 2025 · similarity 0.87)
- Focus On This, Not That! Steering LLMs with Adaptive Feature Specification (ICML 2025 · similarity 0.86)
- Improving Instruction-Following in Language Models through Activation Steering (ICLR 2025 · similarity 0.86)
- Improved Representation Steering for Language Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
