Research map / Large language models / Language, safety and interpretability
Beyond I’m Sorry, I Can’t: Dissecting Large-Language-Model Refusal
Nirmalendu Prakash, Yeo Wei Jie, Amir Abdullah, Ranjan Satapathy, Erik Cambria, Roy Ka-Wei Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| ablate random active features instead of jailbreak-critical features (ablation) | SAE feature ablation · D_harmful | jailbreak count: 0 samples on LLaMA, 5 samples on Gemma | Hurt |
| keep silent (zero-activation) critical features in the ablation set (ablation) | Stage 2 feature set · D_harmful | jailbreak success: drops to 372 (LLaMA) and 103 (Gemma) samples when inactive features removed | Improved |
| rank Stage 3 features with factorization machine instead of linear probe (ablation) | Stage 3 feature discovery · D_harmful, LLaMA | jailbroken samples: 101 samples for linear probe, significantly fewer than FM | Improved |
| FM embedding size 5 vs 20 (ablation) | Factorization machine · Feature embeddings | embedding diversity: cosine similarity > 0.9 at size 5 | Improved |
| FM embedding size 50 instead of 20 (ablation) | Factorization machine · Feature embeddings | embedding quality / training cost | No clear effect |
| add Stage 3 FM-found features to Stage 2 set | Refusal-feature pipeline · D_harmful ASR | ASR: 0.33 -> 0.31 (Gemma), 0.70 -> 0.57 (LLaMA) | Hurt |
| ablate only Stage 3 features (ablation) | Stage 3 features · D_harmful | jailbreak rate: 330/372 (89%) on LLaMA, 8 (3%) on Gemma | Mixed |
| use SAE refusal-feature ablation instead of single refusal direction ablation | LLaMA-3.1-8B-IT · ALPACA CE loss | CE loss: 1.78 -> 2.17 | Hurt |
| use features active on at least k tokens (Phi-3 Mini approach) instead of the proposed pipeline | Baseline feature search · D_harmful | max ASR: 0.1 Gemma, 0.07 LLaMA | Hurt |
| exclude layers 1 and 2 features from candidate set (ablation) | Stage 1 · Candidate selection | response quality | Improved |
| increase K beyond 2000 in Stage 3 (ablation) | LLaMA · Stage 3 top-K ablation | response quality | Hurt |
| clamp redundant features to high values on BOS token | Gemma-2-2B-IT · benign prompts | refusal behaviour | Improved |
Closest papers by meaning
- Refusal Direction is Universal Across Safety-Aligned Languages (NeurIPS 2025 · similarity 0.93)
- Latent-space Attacks for Refusal Evasion in Language Models (NeurIPS 2026 · similarity 0.93)
- Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning (ICML 2025 · similarity 0.92)
- The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence (ICML 2025 · similarity 0.92)
- Refusal in Language Models Is Mediated by a Single Direction (NeurIPS 2024 · similarity 0.92)
- SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
