Research map / Large language models / Language, safety and interpretability

Steering Externalities: Benign Activation Steering Unintentionally Increases Jailbreak Risk for Large Language Models

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Mechanistic analysis of attention and representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that activation steering vectors built from benign data (compliance and JSON-format steering) erode refusal behavior on Llama-2/3 and Gemma, raising harmful-query ASR and amplifying black-box jailbreaks (CoP, PAIR, TAP).

Problem

Activation steering used by developers for benign utility (compliance, JSON formatting) may unintentionally weaken safety alignment and increase jailbreak vulnerability in deployed LLMs.

Approach

Introduces "steering externalities": steering vectors learned from benign datasets (Alpaca compliance contrast pairs, IFEval JSON instructions) are deployed globally and act as a force multiplier for black-box jailbreaks, explained by early-token refusal-gate shifts and harmful-prompt representations moving toward the harmless region.

Key results

• Compliance steering raised Llama-3-8B-Instruct benchmark-only HarmBench ASR from 2% to 38.5%, and JSON steering raised Llama-2-7B-Chat from 0% to 11.5%.
• Combining CoP with compliance steering pushed Llama-2-7B-Chat ASR from 77% to 95%, and JSON steering reached 88.75% on the same model.
• On Llama-2-7B-Chat with 50 HarmBench questions, compliance steering raised PAIR ASR from 10% to 20% and TAP ASR from 20% to 34%.

Surprising findings

• Purely syntactic JSON-format steering, which is not safety-related, also increased harmful-query ASR and jailbreak success.
• Compliance steering reduced SorryBench refusal on Llama-3-8B-Instruct from 86% to 30% under a Mistral judge, even though its harmless-refusal utility improved.

Limitations

• Evaluation limited to three open-weight 7-8B models plus two additional models in an appendix, with small 50-prompt subsets for PAIR/TAP.
• Mechanistic claims rest on t-SNE projections and KL analysis that the paper itself describes as qualitative/illustrative.
• Mitigation STEER-BIND is only preliminarily explored on one model.

Experimental setup

Datasets: Alpaca IFEval HarmBench SorryBench BeaverTails

Benchmarks: HarmBench SorryBench Alpaca IFEval

Metrics: attack success rate refusal rate JSON correctness win-rate per-token KL divergence

Baselines: CoP PAIR TAP Original (unsteered) models

Builds on: CAST (conditional activation steering) Composition-of-Principles PAIR TAP HarmBench

Closest papers by meaning

• SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering (AAAI 2025 · similarity 0.88)
• Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models (ICLR 2025 · similarity 0.87)
• The Sharp Directions Are Against You: Curvature Analysis of Activation Steering (NeurIPS 2026 · similarity 0.87)
• When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search (NeurIPS 2026 · similarity 0.87)
• Steering When Necessary: Flexible Steering Large Language Models with Backtracking (NeurIPS 2025 · similarity 0.87)
• Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.87)