Research map / Large language models / Language, safety and interpretability
The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search
Rongzhe Wei, Peizhi Niu, Xinjie Shen, Tony Tu, Yifan Li, Ruihan Wu, Eli Chien, Pin-Yu Chen, Olgica Milenkovic, Pan Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace static a-priori decomposition (Multi-Agent Jailbreak) with adaptive tree-structured CKA-Agent | decomposition-based jailbreak · HarmBench, four commercial target LLMs | Full Success rate: Claude-Haiku-4.5: 0.786 -> 0.960 | Improved |
| use prompt-optimization attacks against strongly aligned Claude-Haiku-4.5 instead of weaker targets | PAIR · HarmBench | Full Success rate: 0.810 (Gemini-2.5-Flash) -> 0.032 (Claude-Haiku-4.5) | Hurt |
| let CKA-Agent see the full conversation history (single-session CKA-Agent-Branch) instead of cross-session queries (ablation) | CKA-Agent · HarmBench on GPT-oss-120B | Full Success rate: 0.976 -> 0.786 | Hurt |
| let CKA-Agent see the full conversation history (single-session CKA-Agent-Branch) (ablation) | CKA-Agent · HarmBench on Gemini-2.5-Flash | Full Success rate: 0.968 -> 0.921 | No clear effect |
| interactive querying of the target model versus attacker's own knowledge (ablation) | CKA-Agent · HarmBench on Gemini-2.5-Pro | prompts solved (count): CKA Only 26 vs Self Only 2 | Improved |
| apply Llama Guard or Circuit Breaker defenses against optimization-based attacks (ablation) | AutoDAN, PAIR, PAP, TAP, ActorBreaker · HarmBench on Gemini-2.5-Flash | Full Success rate | Improved |
| apply input-level defenses (Llama Guard, rephrasing, perturbation) against decomposition-based attacks (ablation) | CKA-Agent and Multi-Agent Jailbreak · HarmBench on Gemini-2.5-Flash | Full Success rate | No clear effect |
| apply Circuit Breaker representation-level defense against CKA-Agent (ablation) | CKA-Agent · HarmBench on Gemini-2.5-Flash | Full Success rate | Improved |
| apply mutation-based defenses (rephrasing, character perturbation) (ablation) | jailbreak methods · HarmBench on Gemini-2.5-Flash | Full Success rate | No clear effect |
Closest papers by meaning
- $R^2$-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning (ICLR 2025 · similarity 0.88)
- Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization (ICML 2026 · similarity 0.88)
- Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Models (ICLR 2025 · similarity 0.87)
- GuardAgent: Safeguard LLM Agents via Knowledge-Enabled Reasoning (ICML 2025 · similarity 0.87)
- SafeHarbor: Defining Precise Decision Boundaries via Hierarchical Memory-Augmented Guardrail for LLM Agent Safety (ICML 2026 · similarity 0.87)
- HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
