Research map / Adversarial robustness and security / Certified and empirical robustness
Adversarial Déjà Vu: Jailbreak Dictionary Learning for Stronger Generalization to Unseen Attacks
Mahavir Dabas, Tran Huynh, Nikhil Reddy Billa, Jiachen T. Wang, Peng Gao, Charith Peris, Yao Ma, Rahul Gupta, Ming Jin, Prateek Mittal, Ruoxi Jia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on compositions of jailbreak-dictionary skills (ASCoT) instead of no defense | Llama-3.1-8B-Instruct · StrongReject mean harmful score | mean harmful score: 0.38 -> 0.05 | Improved |
| use ASCoT instead of refusal training on plain harmful queries | Llama-3.1-8B-Instruct · StrongReject mean harmful score | mean harmful score: 0.15 -> 0.05 | Improved |
| use ASCoT instead of CAT/LAT latent adversarial training | Llama-3.1-8B-Instruct · XSTest | over-refusal rate: 0.98 -> 0.12 (LAT* vs ASCoT closed) | Improved |
| use ASCoT instead of persuasion-only PAP training | Llama-3.1-8B-Instruct · GALA multi-turn | harmful score: 0.01 -> 0.08 | Mixed |
| train ASCoT with open-weight Qwen3 pipeline instead of GPT-4.1 (ablation) | ASCoT · StrongReject mean harmful score | mean harmful score: 0.05 -> 0.07 | No clear effect |
| train on single-turn data only, test on multi-turn GALA | ASCoT · GALA multi-turn | harmful score: 0.77 -> 0.08 | Improved |
| test on unseen compositions of seen primitives | ASCoT · novel compositions k=2..5 | harmful score: 0.62 -> 0.00 | Improved |
| increase skill coverage (12 to all primitives) at fixed data size (ablation) | ASCoT · PAIR and AutoDAN-Turbo | harmful score | Improved |
| train on shallow compositions (k=1,2) vs deep (k=4,5) (ablation) | ASCoT · PAIR vs AutoDAN-Turbo | harmful score | Mixed |
| compress 14,070 raw skills into 397-atom dictionary (ablation) | Jailbreak Dictionary · unseen attack explanation | explainability score (GPT-4.1): 4.23 -> 4.21 | No clear effect |
| compress raw skills into sparse dictionary (ablation) | Jailbreak Dictionary · unseen attack explanation | active primitives per skill: 12.11 -> 5.90 | Improved |
| move temporal cutoff later (more seen attacks) (ablation) | Jailbreak Dictionary · unseen attack explanation | explainability score | Improved |
Closest papers by meaning
- Adversarial Reasoning at Jailbreaking Time (ICML 2025 · similarity 0.91)
- Short-length Adversarial Training Helps LLMs Defend Long-length Jailbreak Attacks: Theoretical and Empirical Evidence (NeurIPS 2025 · similarity 0.91)
- Fight Back Against Jailbreaking via Prompt Adversarial Tuning (NeurIPS 2024 · similarity 0.90)
- RobustKV: Defending Large Language Models against Jailbreak Attacks via KV Eviction (ICLR 2025 · similarity 0.89)
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.89)
- Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
