Research map / Large language models / Language, safety and interpretability
Fine-Grained Activation Steering: Steering Less, Achieving More
Zijian Feng, Tianjiao Li, Zixiao Zhu, Hanzhang Zhou, Junlang Qian, Li Zhang, Chua Jia Jim Deryl, Mak Lee Onn, Gee Wah Ng, Kezhi Mao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use localized AU-level steering (FFN) instead of SADI head steering | Gemma2-9B-it · Commonsense + math average accuracy | accuracy avg: 82.05 -> 83.96 | Improved |
| AUSteer-FFN vs SADI | LLaMA2-7B-Chat · Commonsense + math average accuracy | accuracy avg: 59.49 -> 61.34 | Improved |
| AUSteer-Head vs SADI | Qwen3-8B · Commonsense + math average accuracy | accuracy avg: 80.80 -> 83.21 | Improved |
| AUSteer on RealToxicPrompts vs SADI | Qwen3-8B · RealToxicPrompts | detox: 78.07 -> 81.65 | Improved |
| AUSteer-FFN on BPO vs SADI | Gemma2-9B-it · BPO | delta WR: 25.50 -> 30.00 | Improved |
| AUSteer-Head vs SADI on detox | Gemma2-9B-it · RealToxicPrompts | detox: 99.08 -> 99.00 | No clear effect |
| use activation-momentum localization instead of random localization (ablation) | AUSteer-FFN Gemma2-9B-it · Commonsense + math average accuracy | accuracy avg: 79.08 -> 83.96 | Improved |
| use activation-momentum localization instead of activation difference (ablation) | AUSteer-FFN Gemma2-9B-it · Commonsense + math average accuracy | accuracy avg: 83.12 -> 83.96 | Improved |
| use adaptive steering vector instead of fixed vector (ablation) | AUSteer-FFN Gemma2-9B-it · Commonsense + math average accuracy | accuracy avg: 82.05 -> 83.96 | Improved |
| use adaptive strength instead of fixed strength (ablation) | AUSteer-FFN Gemma2-9B-it · Commonsense + math average accuracy | accuracy avg: 83.04 -> 83.96 | Improved |
| steer more than 5,000 AUs instead of fewer than 100 (ablation) | AUSteer LLaMA2-7B-Chat · COPA | accuracy: 82.8 -> 70.4 | Hurt |
| steer all 131,072 AUs (ablation) | AUSteer LLaMA2-7B-Chat · commonsense | accuracy | Hurt |
Closest papers by meaning
- Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior (NeurIPS 2026 · similarity 0.92)
- Steering When Necessary: Flexible Steering Large Language Models with Backtracking (NeurIPS 2025 · similarity 0.91)
- Improving Instruction-Following in Language Models through Activation Steering (ICLR 2025 · similarity 0.90)
- Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectors (ICLR 2025 · similarity 0.90)
- Steer2Edit: From Activation Steering to Component-Level Editing (NeurIPS 2026 · similarity 0.89)
- When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
