Research map / Large language models / Language, safety and interpretability

Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation

ICLR 2025 · Large language models › Language, safety and interpretability · approach: Safety alignment of LLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a training-free method that extracts an orthogonalized false-refusal direction from pseudo-harmful and harmless prompts and ablates it from model weights to reduce false refusal while preserving safety and general capability.

Problem

Chat LLMs falsely refuse safe requests that superficially resemble harmful ones (false refusal), and existing mitigations are either training-based (inflexible), training-free with inference overhead, or imprecise (degrade general capability).

Approach

Extract a false-refusal diff-in-means vector from pseudo-harmful vs harmless prompts, partially orthogonalize it against the true-refusal vector (coefficient lambda), and ablate the single selected vector from activations, which can be folded into weights for zero inference overhead and tunable refusal level.

Key results

• Ablating the orthogonalized false refusal vector raises Llama2-7B-Chat XSTest-Safe(H) compliance from 13.6% to 42.4% and ORB-H from 14.8% to 65.6%, with JBB compliance at 5.0% and MMLU changing by under 1 point.
• On Llama2-7B-Chat, the method achieves 0% XSTest-Unsafe compliance versus 6.5% for SCAN, with MMLU 47.2 vs 40.5 and PPL change +0.1 vs +1.56.
• Partial orthogonalization (lambda) gives fine-grained control: lowering lambda increases compliance on pseudo-harmful and, for some models, harmful prompts.

Measured effects reported in the paper

Surprising findings

• Naively ablating the raw diff-in-means false refusal vector also removes refusal on harmful prompts (harmful CR rises to 46.1%), showing the true and false refusal vectors are entangled.
• Adding the true refusal vector increases safety but degrades ARC-C, MMLU and Wikitext perplexity, unlike ablation.

Limitations

• Vector quality depends on the pseudo-harmful data source, mainly OR-Bench-Hard; diversity of pseudo-harmful samples could be improved.
• Vector selection uses first-token refusal score, which ignores sequence-level behavior.
• Lambda-based calibration has no clear ground truth for what counts as safe, so tuning is user-judgment dependent.

Experimental setup

Datasets: AdvBench MaliciousInstruct TDC2023 Alpaca OR-Bench-Hard JailbreakBench XSTest OKTest

Benchmarks: JailbreakBench AdvBench-style harmful test OR-Bench-Hard XSTest-Safe (hard subset) OKTest MMLU ARC-C Wikitext perplexity TruthfulQA

Metrics: compliance rate accuracy perplexity refusal score

Baselines: SCAN SafeDecoding Self-CD DRO system prompt

Builds on: refusal direction (Arditi et al.) difference-in-means SCAN

Closest papers by meaning

• SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal (ICLR 2025 · similarity 0.90)
• Refusal in Language Models Is Mediated by a Single Direction (NeurIPS 2024 · similarity 0.90)
• Refusal Direction is Universal Across Safety-Aligned Languages (NeurIPS 2025 · similarity 0.90)
• Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning (ICML 2025 · similarity 0.90)
• Latent-space Attacks for Refusal Evasion in Language Models (NeurIPS 2026 · similarity 0.90)
• OR-Bench: An Over-Refusal Benchmark for Large Language Models (ICML 2025 · similarity 0.89)