Research map / Large language models / Language, safety and interpretability
Who's asking? User personas and the mechanics of latent misalignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that user-persona manipulation via activation steering (CAA) on Llama 2 13B Chat changes refusal of rewritten adversarial queries more than direct refusal steering, and that harmful content persists in early layers and can be recovered via early decoding.
Problem
Safety-tuned LLMs may refuse harmful queries yet still leak harmful content; the paper asks whether willingness to answer depends on the inferred user persona and what internal mechanisms cause this.
Approach
Treat the model's inferred user persona as a lever: steer persona representations (contrastive activation addition) at a mid layer, compare against prompt-prefix persona manipulation and direct refusal/fulfillment steering, and analyze the mechanism with early decoding, open-ended Patchscopes, and steering-vector cosine geometry.
Key results
- Persona CAA steering at layer 13 shifts response rate by up to about +35% (selfish CAA-), exceeding prompt-prefix persona effects.
- Refusal CAA- only raises the baseline response rate by 7.7%, while refusal and fulfillment prompts have little effect on harmful responses.
- Early decoding from layers above 13 yields 55% aggregated response rate versus 40% at the best single layer (layer 25), suggesting layer-specific, attack-specific safeguards.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Manipulate user persona with activation steering rather than prompt-prefix statements | Llama 2 13B chat · SneakyAdvBench-eval | response rate | Helped |
| Subtract the selfish-persona steering vector at layer 13 | Llama 2 13B chat · SneakyAdvBench-eval | response rate: +35% | Helped |
| Use user-persona steering rather than directly steering refusal behavior | Llama 2 13B chat · SneakyAdvBench-eval | response rate | Helped |
| Prepend fulfillment instructions (ablation) | Llama 2 13B chat · SneakyAdvBench-eval | response rate | No clear effect |
| Subtract the refusal steering vector (ablation) | Llama 2 13B chat · SneakyAdvBench-eval | response rate: 7.7% | Helped |
| Add refusal statements as a prompt prefix (ablation) | Llama 2 13B chat · SneakyAdvBench-eval | response rate | Hurt |
| Add the refusal steering vector (ablation) | Llama 2 13B chat · SneakyAdvBench-eval | response rate | Hurt |
| Add the fulfillment steering vector (ablation) | Llama 2 13B chat · SneakyAdvBench-eval | response rate | Helped |
| Subtract the fulfillment steering vector (ablation) | Llama 2 13B chat · SneakyAdvBench-eval | response rate | Hurt |
| Early-decode generations from layers after layer 13 | Llama 2 13B chat · SneakyAdvBench-eval | response rate: 88% | Helped |
| Combine early decoding across layers rather than use only the best single layer (ablation) | Llama 2 13B chat · SneakyAdvBench-eval | response rate: r_{aggregated}[ED] = 0.55; r_{max-layer}[ED] = 0.4 | Helped |
| Steer user persona at layer 13 (ablation) | Llama 2 13B chat · SneakyAdvBench-eval | response rate: 4% | Helped |
Surprising findings
- Natural language instructions to fulfill harmful requests have almost no effect, while steering persona vectors can raise responsiveness more than direct refusal steering.
- The most effective persona interventions for increasing responsiveness are anti-social personas applied with a negative multiplier, e.g. selfish CAA-.
Limitations
- Main focus is a single model (Llama 2 13B chat); generalization to other models is hypothesized, not verified beyond preliminary Gemma 7B results.
- Autorater agreement with humans is modest (Krippendorff's alpha 0.378 with autorater included), so it is only a reasonable proxy.
- Overall capability effects of the interventions were not studied.
Experimental setup
Datasets: AdvBench SneakyAdvBench
Benchmarks: SneakyAdvBench-eval
Metrics: response rate (auto-rater) Krippendorff's alpha cosine similarity paired t-test
Baselines: Baseline Prompting Prompt prefix (PP) Refusal prompt prefix Fulfillment prompt prefix
Builds on: contrastive activation addition Patchscopes SelfIE AdvBench
Closest papers by meaning
- Persona-Model Collapse in Emergent Misalignment (NeurIPS 2026 · similarity 0.90)
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models (NeurIPS 2026 · similarity 0.88)
- Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs (ICLR 2024 · similarity 0.86)
- The Art of Saying No: Contextual Noncompliance in Language Models (NeurIPS 2024 · similarity 0.86)
- Refusal in Language Models Is Mediated by a Single Direction (NeurIPS 2024 · similarity 0.85)
- Revisiting "Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing" (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.