Research map / Large language models / Language, safety and interpretability
Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes JOOD, a black-box jailbreak that transforms harmful text and image inputs (e.g., word mixing, image mixup) into out-of-distribution variants to bypass RLHF-based safety alignment in LLMs and MLLMs.
Problem
Safety alignment via RLHF may not generalize to harmful inputs that fall outside the aligned training distribution, leaving LLMs and MLLMs vulnerable to jailbreaks, especially proprietary models like GPT-4, GPT-4V, GPT-4o and o1.
Approach
Off-the-shelf data transformations (text-mixing of harmful words with auxiliary words, image mixup/CutMix/ResizeMix of harmful images with auxiliary images) push harmful inputs out of the safety-aligned distribution, raising model uncertainty about malicious intent and enabling black-box jailbreaks; harmfulness is scored by an LLM judge and the max-harm response over sampled transforms is reported
Key results
- JOOD reaches 63% ASR against GPT-4V in the Bombs or Explosives scenario, versus 23% for the best baseline FigStep-Pro.
- On GPT-4 text-only, JOOD achieves 24% ASR in the Firearms scenario where all baselines reached 0%.
- JOOD achieves 83-100% ASR on MiniGPT-4 7B and LLaVA-1.5 13B in the Bombs scenario; with system-prompt defense, JOOD keeps 60% ASR versus 0% for most baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply text-mixing OOD input transformation | GPT-4 with JOOD · Advbench-M Firearms / Weapons | ASR (%): 0 -> 24 | Helped |
| Apply image-mixing OOD input transformation | GPT-4V with JOOD · Advbench-M Bombs or Explosives | ASR (%): 23 -> 63 | Helped |
| Apply image-mixing OOD input transformation | GPT-4V with JOOD · Advbench-M Hacking information | ASR (%): +42% ASR | Helped |
| Replace image-mixing transformations with RandAug non-mixing transformations (ablation) | JOOD image attack on GPT-4V · Advbench-M scenarios | ASR and harmfulness | Hurt |
| Vary the mixup coefficient α (ablation) | JOOD image attack on GPT-4V · Advbench-M Bombs or Explosives | ASR and harmfulness | Mixed |
| Mix the harmful image with a semantically more similar auxiliary image (ablation) | JOOD image attack on GPT-4V · Advbench-M Bombs or Explosives | harmfulness | Hurt |
| Use typographic auxiliary images instead of realistic auxiliary images (ablation) | JOOD image attack on GPT-4V · Advbench-M scenarios | harmfulness | Helped |
| Apply system-prompt-based defense (ablation) | JOOD on GPT-4V · Advbench-M Bombs or Explosives | ASR (%): 63% -> 60% | Hurt |
Surprising findings
- Image mixup with highly similar auxiliary images (e.g., Molotov cocktail) reduces harmfulness, while dissimilar auxiliaries increase it.
- Typographic auxiliary images amplify harmfulness of mixup attacks more than realistic auxiliary images.
Limitations
- Evaluation relies on LLM-based harmfulness scoring and binary judging, which may be imperfect.
- Attack success depends on sampling many transformed variants and reporting the max-harm response, which is a best-of-n style evaluation.
- Only simple off-the-shelf transforms were explored; more advanced OOD transformations were not studied.
Experimental setup
Datasets: AdvBench AdvBench-M
Benchmarks: AdvBench AdvBench-M
Metrics: harmfulness score (HF, 0-10) attack success rate (ASR) refusal rate
Baselines: Vanilla CipherChat PAIR FigStep FigStep-Pro HADES
Builds on: FigStep HADES PAIR CipherChat mixup
Closest papers by meaning
- SafePTR: Token-Level Jailbreak Defense in Multimodal LLMs via Prune-then-Restore Mechanism (NeurIPS 2025 · similarity 0.93)
- Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment (CVPR 2025 · similarity 0.93)
- h4rm3l: A Language for Composable Jailbreak Attack Synthesis (ICLR 2025 · similarity 0.91)
- Safety Alignment Should be Made More Than Just a Few Tokens Deep (ICLR 2025 · similarity 0.91)
- Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions (ICML 2025 · similarity 0.91)
- Lifelong Safety Alignment for Language Models (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.