Research map / Large language models / Language, safety and interpretability
Adjacent Words, Divergent Intents: Jailbreaking Large Language Models via Task Concurrency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes word-level task concurrency for LLMs (interleaving a harmful and a benign task word by word) and JAIL-CON, an iterative black-box jailbreak that uses CVT and CIT variants and a GPT-4o-mini shadow judge to obtain harmful answers that evade moderation filters.
Problem
LLMs are jailbroken mainly via sequential task interactions, and guardrails may miss harmful content hidden in concurrent (interleaved) tasks; the paper asks whether concurrency introduces new safety vulnerabilities.
Approach
Interleave a harmful task with a benign auxiliary task at the word level, separated by delimiters, so adjacent words carry divergent intents; the target LLM answers both interleaved tasks (CVT) or answers only the harmful one while emitting idle placeholders (CIT), and an iterative loop swaps auxiliary tasks until a judge-scored harmful answer is obtained.
Key results
- JAIL-CON achieves average ASR-O of 0.95 across six LLMs without a guardrail, versus 0.71 for the second-best method (GPTFuzzer).
- With the OpenAI Moderation guardrail, JAIL-CON reaches average FR of 0.33 and ASR-E of about 0.62 on open models, with the best ASR-E of 0.76 on GPT-4o.
- Filter rate for harmful-plus-benign concurrent prompts (B+H) drops to 0.37 on average versus 0.71 for harmful-only (H) prompts on OpenAI Moderation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace sequential answering with word-level concurrent task processing (CVT + CIT) | GPT-4o · GSM8K | accuracy: 0.9538 -> 0.9272 | Hurt |
| Combine a benign task with a harmful task, with the benign task first | OpenAI Moderation API · JailbreakBench concurrent prompts | filter rate: 0.71 -> 0.37 | Hurt |
| Use JAIL-CON rather than the strongest reported average-ASR-O baseline, GPTFuzzer | JailbreakBench, 6 evaluated LLMs · JailbreakBench | ASR-O: 0.71 -> 0.95 | Helped |
| Use both CVT and CIT rather than CVT only (ablation) | JAIL-CON · JailbreakBench | ASR-O: 0.79 -> 0.95 | Helped |
| Use both CVT and CIT rather than CIT only (ablation) | JAIL-CON · JailbreakBench | ASR-O: 0.91 -> 0.95 | Helped |
| Replace JailbreakBench auxiliary tasks with TruthfulQA auxiliary tasks (ablation) | JAIL-CON on LLaMA2-13B · JailbreakBench | ASR-O: 0.86 -> 0.92 | Helped |
| Remove the default prompt template (ablation) | JAIL-CON on GPT-4o · JailbreakBench | ASR-O: 0.95 -> 0.93 | Hurt |
| Increase the iteration budget (ablation) | JAIL-CON · JailbreakBench | ASR-O | Helped |
| Change the separator choice (ablation) | JAIL-CON · JailbreakBench | ASR-O: typically within ±0.02 | No clear effect |
| Use different auxiliary-task sources (ablation) | JAIL-CON · JailbreakBench, most evaluated LLMs | ASR-O: ±0.02 | No clear effect |
| Add chat history before the attack (ablation) | JAIL-CON on GPT-4o · JailbreakBench, multi-turn dialogue | ASR-O: within ±0.03 | No clear effect |
Surprising findings
- Combining a harmful task with a benign one substantially reduces guardrail filtering, even though LLMs still answer the benign task well.
- CIT-only (answering just the harmful task with idle placeholders) performs nearly as well as the full two-variant attack while halving queries.
Limitations
- Attack relies on a closed-source shadow judge (GPT-4o mini) and a GPT-4o evaluation judge, so results depend on judge choice.
- Separator choice and word-level splitting are heuristic; separator optimization left for future work.
- Evaluated on a fixed set of JailbreakBench harmful tasks with temperature 0.
Experimental setup
Datasets: JailbreakBench GSM8K TruthfulQA
Benchmarks: JailbreakBench AdvBench HarmBench GSM8K TruthfulQA
Metrics: ASR-O FR (filter rate) ASR-E accuracy
Baselines: GCG Base64 Combination PAIR GPTFuzzer FlipAttack JAM TAP
Builds on: StrongREJECT JailbreakBench PAIR GPTFuzzer
Closest papers by meaning
- An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks (ICML 2025 · similarity 0.89)
- Multi-Turn Jailbreaking Large Language Models via Attention Shifting (AAAI 2025 · similarity 0.89)
- Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Models (ICLR 2025 · similarity 0.89)
- Many-shot Jailbreaking (NeurIPS 2024 · similarity 0.88)
- Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs (NeurIPS 2025 · similarity 0.88)
- Weak-to-Strong Jailbreaking on Large Language Models (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.