Research map / Vision-language and multimodal / Multimodal LLMs
Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Jailbreak-AudioBench, a toolbox, dataset (~158K audio samples from TTS-converted harmful questions with 20 hidden-semantic audio edits), and benchmark evaluating nine LALMs for audio editing jailbreak vulnerability, plus a query-based editing attack and a spoken-instruction defense.
Problem
Large audio language models may be jailbroken through audio-specific hidden semantics (emphasis, speed, intonation, tone, noise, accent, emotion), but this vulnerability has been little studied for end-to-end LALMs.
Approach
Use a TTS-plus-audio-editing toolbox to inject paralinguistic hidden semantics into harmful spoken questions, build explicit/implicit jailbreak audio datasets from existing harmful-question benchmarks, and benchmark LALM robustness, with a combinatorial query-based editing attack that searches over 32 edit combinations.
Key results
- Qwen2-Audio ASR on explicit original audio rose from 13.3% to 48.8% under query-based audio editing; SALMONN-7B rose from 31.6% to 85.1%.
- SALMONN-7B is most vulnerable to editing (e.g., +21.4% ASR with emphasis volume x10 on explicit subtype), while GPT-4o-Audio shows ASR changes under 1.7% for most edits.
- Llama Guard 3 judgments match human volunteer ASR within about 4.5% (human ASR higher), and Llama Guard 3 is adopted as the final criterion.
- Prepending a spoken refusal instruction reduces ASR across models but residual ASR remains high.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Query 32 audio-editing variants per sample instead of using the original audio alone | Qwen2-Audio-7B · Explicit Small | ASR: 13.3% to 48.8% | Hurt |
| Query 32 audio-editing variants per sample instead of using the original audio alone | SALMONN-7B · Explicit Small | ASR: 31.6% to 85.1% | Hurt |
| Query 32 audio-editing variants per sample instead of using the original audio alone | GPT-4o-Audio · Explicit Small | ASR: 0.7% to 8.4% | Hurt |
| Query 32 audio-editing variants per sample instead of using the original audio alone | Gemini-2.5-Flash · Explicit Small | ASR: 8.1% to 49.4% | Hurt |
| Query 32 audio-editing variants per sample instead of using the original audio alone | BLSP · Explicit Small | ASR: 48.1% to 87.8% | Hurt |
| Query 32 audio-editing variants per sample instead of using the original audio alone | SpeechGPT · Explicit Small | ASR: 12.2% to 42.4% | Hurt |
| Query 32 audio-editing variants per sample instead of using the original audio alone | VITA-1.5 · Explicit Small | ASR: 3.4% to 47.7% | Hurt |
| Query 32 audio-editing variants per sample instead of using the original audio alone | MiniCPM-o-2.6 · Explicit Small | ASR: 16.8% to 65.7% | Hurt |
| Add emphasis editing (Volume*10) | SALMONN-7B · Explicit Subtype | ASR change: +21.4% | Hurt |
| Add machine background noise | SALMONN-7B · Explicit Subtype | ASR change: +20.3% | Hurt |
| Add celebrity voice conversion (Kanye West) | SALMONN-7B · Explicit Subtype | ASR change: +12.8% | Hurt |
| Add emotion editing (Scream) | SALMONN-7B · Explicit Subtype | ASR change: +18.0% | Hurt |
Surprising findings
- Closed-source GPT-4o-Audio is largely robust to edits, yet query-based editing raises its ASR from 0.7% to 8.4%.
- Layer-wise t-SNE shows robust Qwen2-Audio converges edited audio toward semantic clusters by later layers, while SALMONN-7B keeps editing-based clusters through all layers.
Limitations
- Llama Guard 3 can mark responses that merely repeat the harmful prompt as successful attacks.
- Uses TTS-generated speech rather than natural human recordings, limiting realism of prosody and speaker variation.
- Closed-source model evaluation was limited to smaller subsets due to API cost.
Experimental setup
Datasets: AdvBench MM-SafetyBench RedTeam-2K SafeBench
Benchmarks: Jailbreak-AudioBench Explicit Subtype Jailbreak-AudioBench Implicit Subtype Explicit Small
Metrics: attack success rate
Baselines: BLSP SpeechGPT Qwen2-Audio SALMONN VITA-1.5 R1-AQA MiniCPM-o-2.6 GPT-4o-Audio Gemini-2.5-Flash
Builds on: AdvBench MM-SafetyBench RedTeam-2K SafeBench Llama Guard 3
Closest papers by meaning
- ALMGuard: Safety Shortcuts and Where to Find Them as Guardrails for Audio–Language Models (NeurIPS 2025 · similarity 0.91)
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.89)
- MMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Vision Language Models (AAAI 2025 · similarity 0.89)
- Distraction is All You Need for Multimodal Large Language Model Jailbreaking (CVPR 2025 · similarity 0.89)
- Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy (CVPR 2025 · similarity 0.88)
- Multi-Turn Jailbreaking Large Language Models via Attention Shifting (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.