Research map / Large language models / Language, safety and interpretability

The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions

ICML 2025 · Large language models › Language, safety and interpretability · approach: Safety alignment of LLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyzes the residual space of activation shifts from safety fine-tuning (SSFT and DPO) on Llama 3.1 8B to show refusal is governed by a dominant direction plus interpretable non-dominant directions, and uses this to craft trigger-removal jailbreaks.

Problem

Prior mechanistic work models LLM safety behavior with a single probe/refusal direction, which conflates multiple features and hides how safety fine-tuning shapes multiple interacting directions and vulnerabilities to jailbreaks.

Approach

Defines a safety residual space as the linear span of activation shifts induced by safety fine-tuning, extracts its SVD components as feature directions, and introduces Partial Layer-wise Relevance Propagation (PLRP) to interpret each direction through its top-contributing input tokens and cross-layer relevance.

Key results

• A dominant direction (L25-C1) predicts refusal behavior with high accuracy in later layers, and removing it eliminates the fine-tuned model's refusal ability.
• Removing L14-C6 selectively ablates refusal of PAIR-style jailbreaks while preserving handling of other attack types.
• Trigger Removal attack keeps about 40% attack success after 80-shot per-jailbreak fine-tuning, while PAIR, Flip and ReNeLLM success drops to near zero.

Measured effects reported in the paper

Surprising findings

• Trigger Removal attacks remain effective after fine-tuning on jailbreak examples, whereas other jailbreak methods are quickly neutralized.
• DPO trained directly on the original model learns more divergent directions and inconsistent PAIR refusal, while SFT-initialized DPO behaves more consistently.

Limitations

• Analysis is limited to a small preference dataset of 2600 samples and a single refusal-focused setting, and the authors note interpretation becomes harder with larger data and models.
• Relies on a linear approximation of safety fine-tuning shifts, with some directions flipping across layers and degrading generation when extended.
• Interpretations of directions depend on token-level PLRP attributions and GPT-4o labeling, which are qualitative.

Experimental setup

Datasets: STRONG REJECT OR-Bench AdvBench Alpaca

Benchmarks: STRONG REJECT OR-Bench Alpaca perplexity

Metrics: StrongReject score refusal accuracy attack pass rate effective rank MSE perplexity

Baselines: refusal direction probe Logit Lens Best-of-N BASE GPTFuzz Flip PAIR ReNeLLM GCG

Builds on: Linear Representation Hypothesis Layer-wise Relevance Propagation refusal direction DPO STRONG REJECT

Closest papers by meaning

• Fail-Closed Alignment for Large Language Models (NeurIPS 2026 · similarity 0.91)
• Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons (NeurIPS 2025 · similarity 0.91)
• Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.90)
• Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications (ICML 2024 · similarity 0.90)
• LLMs Encode Harmfulness and Refusal Separately (NeurIPS 2025 · similarity 0.90)
• The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence (ICML 2025 · similarity 0.90)