Research map / Adversarial robustness and security / Backdoor and poisoning
Obfuscated Activations Bypass LLM Latent-Space Defenses
Luke Bailey, Alex Serrano, Abhay Sheshadri, Mikhail Seleznyov, Jordan Taylor, Erik Jenner, Jacob Hilton, Stephen Casper, Carlos Guestrin, Scott Emmons
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add an obfuscation loss to embedding suffix attack | probe-evading jailbreak · Harmfulness probes | Recall @ 1% FPR (%): 100% -> 0% | Improved |
| use generation-based instead of instruction-based mean difference probe | jailbreak detection · Jailbreak detection | Recall @ 1% FPR (%): 39.66 -> 71.94 | Improved |
| use MLP instead of mean difference probe (generation-based) | probe · Jailbreak detection | Recall @ 1% FPR (%): 71.94 -> 96.29 | Improved |
| train obfuscation suffix against OOD detector | input-based probes · Obfuscation transfer | Recall @ 1% FPR (%): bypasses input-based probes | Improved |
| use circuit breakers training | Llama-3-8B-Instruct · Embedding suffix attack | Jailbreak success rate (%): 95% | Hurt |
| add obfuscated adversarial training (OAT) | Llama-3-8B-Instruct · Obfuscation suffix attack | Jailbreaking rate (%): 95% -> 35% | Improved |
| add obfuscated adversarial training (OAT) | Llama-3-8B-Instruct · Obfuscation suffix attack | Gradient steps to reach zero recall: 256 -> 16,384 | Improved |
| add adversarial training of monitors for 70 epochs | MLP probe · Harmfulness monitoring | Recall on new attack | No clear effect |
| add obfuscation loss in backdoor training (OOD and probes) | backdoored model · Backdoor obfuscation | Attack success rate (%): 95.5 -> 94.9 | No clear effect |
| add obfuscation loss in backdoor training (OOD and probes) | backdoored model · Backdoor obfuscation | False trigger rate (%): 0.64 -> 0.96 | No clear effect |
| raise SAE raw latent probe threshold from 1% to 2% FPR (ablation) | SAE raw latent probe · SQL detection with obfuscation suffix | Recall (%): 0% -> 100% | Improved |
| increase number of poisoned samples (ablation) | backdoor obfuscation · Backdoor with simple trigger | obfuscation vs all detectors: around 200 poisoned samples | Improved |
Closest papers by meaning
- Reliable Weak-to-Strong Monitoring of LLM Agents (ICLR 2026 · similarity 0.85)
- Monitoring Decomposition Attacks with Lightweight Sequential Monitors (ICLR 2026 · similarity 0.85)
- How does information access affect LLM monitors' ability to detect sabotage? (ICML 2026 · similarity 0.84)
- Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space (NeurIPS 2024 · similarity 0.84)
- Bits Leaked per Query: Information-Theoretic Bounds for Adversarial Attacks on LLMs (NeurIPS 2025 · similarity 0.84)
- Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitors (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
