Research map / Architectures and efficiency / RNNs and normalization
Angular Steering: Behavior Control via Rotation in Activation Space
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Angular Steering, which controls LLM behavior (refusal, plus emotion as a secondary use case) by rotating activations within a fixed 2D plane spanned by a feature direction and a PCA-derived axis, with an adaptive variant that only rotates positively aligned activations.
Problem
Activation steering via addition or directional ablation is sensitive to hand-tuned coefficients and activation norms, offers limited granularity, and can degrade coherence and general capabilities when steering interferes with unrelated features.
Approach
Reformulate activation steering as a norm-preserving rotation toward a target angle within a fixed 2D steering plane spanned by a refusal direction (difference-in-means, chosen by cross-layer cosine similarity) and the first principal component of candidate directions; show addition and orthogonalization are special cases of this rotation.
Key results
- Adaptive Angular Steering on Qwen2.5-7B-Instruct reaches HarmBench 0.8750 and LlamaGuard3 1.0000 harmful scores with substring-matching refusal dropping to 0.0000, versus 0.0192 and 0.0000 with no steering.
- Steering in the PC0-based plane keeps perplexity low and stable (mean diff 0.0643 on Qwen2.5-7B) compared with steering in the activation-feature plane (mean diff 2.3969).
- TinyBenchmarks accuracy is largely preserved across the steering circle for most models, with a notable drop for Qwen2.5-3B-Instruct between 160 and 280 degrees.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Adaptive Angular Steering rather than no steering | Qwen2.5-3B-Instruct · AdvBench harmful test | HarmBench score: 0.0000 -> 0.6635 | Helped |
| Apply Adaptive Angular Steering rather than Activation Addition | Qwen2.5-7B-Instruct · AdvBench harmful test | LlamaGuard3 score: 0.9808 -> 1.0000 | Helped |
| Apply Adaptive Angular Steering rather than directional ablation | Qwen2.5-14B-Instruct · AdvBench harmful test | HarmBench score: 0.0288 -> 0.7212 | Helped |
| Use the proposed fixed PCA-based steering plane instead of Span(h, feature) (ablation) | Angular Steering · Steered generations across tested models and steering angles | perplexity: 5.6141 -> 2.9303 (Qwen2.5-3B-Instruct mean) | Helped |
| Use the proposed fixed PCA-based steering plane instead of Span(h, feature) (ablation) | Angular Steering · TinyBenchmarks across steering angles | benchmark performance | Helped |
| Use a random steering plane instead of the target-feature plane (ablation) | Adaptive Angular Steering · Refusal steering, five of six tested models | refusal control | Hurt |
| Add the adaptive alignment mask to non-adaptive Angular Steering (ablation) | Angular Steering · Steered generations, 3B models | perplexity | Helped |
| Use non-adaptive rather than adaptive steering (ablation) | Angular Steering on 3B models · Steered generations | perplexity | Hurt |
Surprising findings
- Non-adaptive angular steering breaks coherence on the 3B models, with refusal phrases appearing in Chinese or other languages.
- Harmful generations have lower perplexity than refusal responses, suggesting alignment masks rather than removes harmful behavior.
Limitations
- Steering plane is chosen heuristically and may not generalize across behaviors or architectures.
- Smaller models (3B) suffer feature interference and incoherent outputs under non-adaptive steering.
- Refusal substring matching does not assess coherence or harmfulness on its own.
Experimental setup
Datasets: AdvBench Alpaca TinyBenchmarks
Benchmarks: TinyBenchmarks (ARC, MMLU, WinoGrande, GSM8k, TruthfulQA, HellaSwag) AdvBench held-out split
Metrics: refusal substring-matching score HarmBench classifier harmful score LlamaGuard3 harmful score perplexity accuracy
Baselines: activation addition directional ablation no steering
Builds on: activation addition directional ablation difference-in-means refusal direction
Closest papers by meaning
- The Sharp Directions Are Against You: Curvature Analysis of Activation Steering (NeurIPS 2026 · similarity 0.90)
- Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior (NeurIPS 2026 · similarity 0.87)
- Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization (NeurIPS 2024 · similarity 0.87)
- When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search (NeurIPS 2026 · similarity 0.86)
- To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Models (ICML 2025 · similarity 0.85)
- Improved Representation Steering for Language Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.