Research map / Adversarial robustness and security / Attacks on text and graphs
AdvWave: Stealthy Adversarial Jailbreak Attack against Large Audio-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AdvWave, the first jailbreak framework for large audio-language models, combining dual-phase gradient optimization over audio tokens, adaptive target search, and classifier-guided urban-noise stealth, plus an LLM-refinement black-box attack on GPT-4o-S2S.
Problem
Jailbreaking large audio-language models (LALMs) via adversarial audio is hard due to gradient shattering from discretized audio tokenization, behavioral variability of LALM responses, and the need for perceptually stealthy audio perturbations.
Approach
A dual-phase optimization first optimizes discrete audio tokens with an adversarial loss, then optimizes the waveform with a triplet retention loss to match those tokens; adaptive target search derives per-query affirmative response prefixes from detoxified queries; and a noise-classifier loss (via Qwen2-Audio) makes the adversarial suffix resemble car horns or similar urban sounds.
Key results
- On average across SpeechGPT, Qwen2-Audio and Llama-Omni, AdvWave reaches ASR-W 0.838 and ASR-L 0.746 versus 0.536 and 0.322 for the best baseline (AutoDAN-Trans).
- On GPT-4o-S2S, AdvWave achieves ASR-W 0.891 and ASR-L 0.994 on HEXPHI-Audio, versus 0.503 and 0.470 for AutoDAN-Trans.
- AdvWave keeps stealthiness scores around 0.70-0.72 across models, while baseline transfer attacks score 0.23-0.49 (vanilla audio 1.0).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace transfer-based jailbreaks with AdvWave | SpeechGPT on AdvBench-Audio · SpeechGPT | ASR-W: 0.179 -> 0.643 | Helped |
| Replace transfer-based jailbreaks with AdvWave | Qwen2-Audio on AdvBench-Audio · Qwen2-Audio | ASR-L: 0.723 -> 0.884 | Helped |
| Replace transfer-based jailbreaks with AdvWave | Llama-Omni on AdvBench-Audio · Llama-Omni | ASR-L: 0.546 -> 0.751 | Helped |
| Replace transfer-based jailbreaks with AdvWave | GPT-4o-S2S API on HEXPHI-Audio · HEXPHI-Audio, average categories | ASR-L: 0.470 -> 0.994 | Helped |
| Replace fixed target "Sure!" with adaptive target search (ablation) | AdvWave · SpeechGPT, Qwen2-Audio, and Llama-Omni | ASR | Helped |
| Add classifier-guided stealthiness penalty (ablation) | AdvWave on Qwen2-Audio · Qwen2-Audio | S_stealth | Helped |
| Add classifier-guided stealthiness penalty (ablation) | AdvWave on Qwen2-Audio · Qwen2-Audio | ASR-L | No clear effect |
| Replace mean-square retention loss with triplet retention loss (ablation) | AdvWave dual-phase optimization · LALM jailbreak optimization | optimization effectiveness | Helped |
| Use different environmental noise targets under classifier guidance | AdvWave · Different environmental noise targets | jailbreak performance | No clear effect |
| Refine black-box prompts iteratively using judge feedback | AdvWave on GPT-4o-S2S API · GPT-4o-S2S API | ASR | Helped |
Surprising findings
- Vanilla unmodified audio already yields high ASR-W (0.928) on Llama-Omni, showing that model safety varies sharply across LALMs.
- AutoDAN-Trans achieves ASR-W 0.648 on Qwen2-Audio but only 0.004 on SpeechGPT, indicating strong model-dependent transfer.
Limitations
- Harmfulness of audio outputs is judged on text transcripts with an LLM judge, since audio-based judgment was found imprecise.
- Stealth is partly evaluated by human judgment and waveform metrics, which are subjective and limited.
- Black-box attack relies on OpenAI TTS and LLM-based prompt refinement, and white-box experiments are limited to three open LALMs.
Experimental setup
Datasets: AdvBench HEXPHI
Benchmarks: AdvBench-Audio HEXPHI-Audio
Metrics: ASR-W ASR-L S_stealth NSR Mel-Sim human stealthiness score
Baselines: Vanilla GCG-Trans BEAST-Trans AutoDAN-Trans LALM-Trans
Builds on: GCG BEAST AutoDAN classifier guidance
Closest papers by meaning
- ALMGuard: Safety Shortcuts and Where to Find Them as Guardrails for Audio–Language Models (NeurIPS 2025 · similarity 0.87)
- Jailbreak-AudioBench: In-Depth Evaluation and Analysis of Jailbreak Threats for Large Audio Language Models (NeurIPS 2025 · similarity 0.86)
- Fast Adversarial Attacks on Language Models In One GPU Minute (ICML 2024 · similarity 0.86)
- Rethinking Audio-Visual Adversarial Vulnerability from Temporal and Modality Perspectives (ICLR 2025 · similarity 0.85)
- LARGO: Latent Adversarial Reflection through Gradient Optimization for Jailbreaking LLMs (NeurIPS 2025 · similarity 0.85)
- Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.