Research map / Large language models / Language, safety and interpretability

The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence

ICML 2025 · Large language models › Language, safety and interpretability · approach: Safety alignment of LLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes gradient-based Refusal Direction Optimization (RDO) and multi-dimensional refusal concept cones, plus a representational independence criterion, showing refusal is mediated by multiple independent directions rather than one.

Problem

Prior work assumes refusal in LLMs is mediated by a single linear direction in activation space; the geometry and independence of refusal mechanisms, and how they can be exploited for jailbreaks, remain poorly understood.

Approach

Instead of difference-in-means on paired prompts, directly optimize a refusal direction by gradient descent with ablation, addition, and retain losses; extend to orthonormal basis defining polyhedral concept cones; and define representational independence via cosine similarity under intervention across layers, not just Euclidean orthogonality.

Key results

• RDO directions achieve jailbreak ASR of about 80% on Gemma 2 2B (79.9 on JailbreakBench with directional ablation) comparable to DIM, while improving TruthfulQA retention (e.g., Gemma 2 2B 51.4 vs 47.8 for DIM, baseline 55.8).
• Refusal mediating concept cones of up to five dimensions are identified across tested models; ASR for best-of-N sampling from a 4-dimensional Gemma 2 2B cone plateaus at four dimensions.
• Orthogonal refusal directions are not representationally independent: ablating an orthogonal RDO direction reduces the DIM direction's representation in later layers; representationally independent directions can be found and ablating top-k of them increases ASR monotonically, surpassing DIM for k>=

Measured effects reported in the paper

Surprising findings

• Orthogonality alone does not imply independence: ablating one orthogonal direction indirectly reduces the representation of the other in later layers.
• For many Qwen 2.5 models, most DIM directions have too high side effects to be usable as attacks.

Limitations

• All refusal directions are optimized on the same targets, which may limit capture of fully distinct mechanisms.
• Existence of higher-dimensional cones is shown, but other undiscovered refusal-mediating regions cannot be ruled out.
• Results are primarily on chat models of 1.5B-14B size from three families; high variance across RepInd runs requires multiple candidate selection.

Experimental setup

Datasets: SALAD-Bench Alpaca JailbreakBench StrongREJECT SORRY-Bench XSTest TruthfulQA MMLU ARC GSM8K

Benchmarks: JailbreakBench StrongREJECT SORRY-Bench TruthfulQA MMLU ARC-Challenge GSM8K XSTest

Metrics: attack success rate refusal score benchmark accuracy change

Baselines: DIM (difference-in-means refusal direction) GCG

Builds on: difference-in-means refusal direction directional ablation activation addition GCG

Closest papers by meaning

• Refusal Direction is Universal Across Safety-Aligned Languages (NeurIPS 2025 · similarity 0.93)
• Latent-space Attacks for Refusal Evasion in Language Models (NeurIPS 2026 · similarity 0.92)
• Refusal in Language Models Is Mediated by a Single Direction (NeurIPS 2024 · similarity 0.91)
• The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions (ICML 2025 · similarity 0.90)
• Fail-Closed Alignment for Large Language Models (NeurIPS 2026 · similarity 0.90)
• LLMs Encode Harmfulness and Refusal Separately (NeurIPS 2025 · similarity 0.90)