Research map / Adversarial robustness and security / Attacks on text and graphs
Towards Robust Multimodal Large Language Models Against Jailbreak Attacks
Ziyi Yin, Yuanpu Cao, Han Liu, Ting Wang, Jinghui Chen, Fenglong Ma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adversarial tuning with SafeMLLM | MLLMs (6 models) · ImgJP (AdvBench) | ASR: 51.33 -> 5.17 | Improved |
| adversarial tuning with SafeMLLM | MLLMs (6 models) · GCG (AdvBench) | ASR: 33.83 -> 0 | Improved |
| adversarial tuning with SafeMLLM instead of CAT | MLLMs (6 models) · FigStep (SafeBench-Tiny) | ASR: 19.33 -> 1.00 | Improved |
| SafeMLLM instead of VLGuard safety tuning | LLaVA-1.5-7B · ImgJP (AdvBench) | ASR: 88.00 -> 6.00 | Improved |
| SafeMLLM instead of VLGuard | LLaVA-1.5-7B · FigStep | ASR: 2.00 -> 0.00 | No clear effect |
| add contrastive loss to attack and defense (ablation) | SafeMLLM · ImgJP (AdvBench) | ASR: 23.00 -> 0.00 | Improved |
| add utility loss on benign pairs (ablation) | SafeMLLM · MM-Vet | score: 21.6 -> 37.8 | Improved |
| add utility loss on benign pairs (ablation) | SafeMLLM · MM-Vet (MiniGPT-4) | score: 7.2 -> 22.8 | Improved |
| add Ph0 and Pt0 token embedding perturbations (vs removing) (ablation) | SafeMLLM · ImgJP, LLaVA-1.5 | ASR | No clear effect |
| token-embedding perturbation instead of adversarial image (ablation) | SafeMLLM · LLaVA-1.5-7B | runtime per iteration: 84.42 -> 20.73 | Improved |
| token-embedding perturbation instead of latent adversarial training (ablation) | SafeMLLM · LLaVA-1.5-13B | runtime per iteration: 192.39 -> 38.70 | Improved |
Closest papers by meaning
- SafePTR: Token-Level Jailbreak Defense in Multimodal LLMs via Prune-then-Restore Mechanism (NeurIPS 2025 · similarity 0.91)
- MMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Vision Language Models (AAAI 2025 · similarity 0.90)
- Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment (CVPR 2025 · similarity 0.89)
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.89)
- MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models (NeurIPS 2024 · similarity 0.89)
- Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models? (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
