Research map / Large language models / Language, safety and interpretability

SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering

AAAI 2025 · Large language models › Language, safety and interpretability · approach: Safety alignment of LLMs · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SCANS, a training-free activation steering method that adds/subtracts a refusal steering vector in middle layers, with a hidden-state-transition similarity classifier choosing the direction, to reduce exaggerated safety while keeping safety and capability.

Problem

Safety-aligned LLMs exhibit exaggerated safety, refusing benign queries that share vocabulary with harmful ones, which limits helpfulness; existing fixes are either training-based with poor results on such queries or decoding/prompt-based with added cost.

Approach

Extract refusal steering vectors as the mean activation difference between harmful and benign queries, anchor steering to middle layers via vocabulary projection, and decide steering direction per query by cosine similarity of a hidden-state transition (query vs. query+"Sure") to a harm reference direction.

Key results

• On XSTest with Llama2-7b-chat, SCANS reaches average 92.00 vs. 67.77 default and 90.66 for Self-CD; OKTest safe refusal drops to 0.33%.
• Across four models SCANS reduces false refusal by 24.7% on XSTest and 26.3% on OKTest on average, and achieves 97.85 overall average vs. 90.71 default.
• Steering-direction classifier reaches F1 0.911 on XSTest, second only to GPT-4 (0.921), and outperforms Llama Guard (0.819).

Measured effects reported in the paper

Surprising findings

• Steering former layers corrupts generation entirely (perplexity in the thousands), while middle layers preserve fluency.
• Too large steering multiplier (alpha=10) produces repetitive nonsense outputs.

Limitations

• Classification errors cause slight drops in refusal on unsafe prompts compared to the original model (e.g., XSTest-Unsafe 93.50 vs 99.00 default for Llama2-7b).
• Larger-model comparisons limited; most baselines not reproduced on 32B+ models.
• Requires choosing steering layers, multiplier alpha, and threshold T per model, with a sensitivity to these hyperparameters.

Experimental setup

Datasets: AdvBench TruthfulQA XSTest OKTest RepE-Data Malicious MMLU XSum WikiText-2 C4 OR-Bench

Benchmarks: XSTest OKTest RepE-Data AdvBench Malicious TruthfulQA MMLU XSum WikiText-2 C4 OR-Bench

Metrics: refusal rate average safe/unsafe accuracy ROUGE-1/2/L accuracy perplexity precision/recall/F1 inference speed GPU memory

Baselines: Prompt Self-CD SafeDecoding DRO Llama Guard GradSafe GPT-4 OpenAI Moderation API Perspective API

Builds on: mean difference steering representation engineering contrastive decoding (Self-CD) SafeDecoding

Closest papers by meaning

• Safety Layers in Aligned Large Language Models: The Key to LLM Security (ICLR 2025 · similarity 0.89)
• The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions (ICML 2025 · similarity 0.89)
• Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons (NeurIPS 2025 · similarity 0.89)
• Safety Alignment Should be Made More Than Just a Few Tokens Deep (ICLR 2025 · similarity 0.89)
• STAIR: Improving Safety Alignment with Introspective Reasoning (ICML 2025 · similarity 0.89)
• Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.89)