Research map / Adversarial robustness and security / Attacks on text and graphs
Activation Manipulation Attack: Penetrating and Harmful Jailbreak Attack Against Large Vision-Language Models
Haojie Hao, Jiakai Wang, Aishan Liu, Yuqing Ma, Haotong Qin, Yuanfang Guo, Xianglong Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add attention shifting loss (ablation) | ActMan on MiniGPT4 · AdvBench | ASR (%): 6.54 -> 54.62 | Improved |
| add attention shifting loss (ablation) | ActMan on LLaVA · AdvBench | ASR (%): 3.85 -> 49.42 | Improved |
| add abnormal pattern concealment loss (ablation) | ActMan on MiniGPT4 · AdvBench | ASR (%): 6.54 -> 58.65 | Improved |
| combine both losses (full visual camouflage) (ablation) | ActMan on MiniGPT4 · AdvBench | ASR (%): 54.62 -> 64.23 | Improved |
| add framing-guided induction (ablation) | ActMan on MiniGPT4 · AdvBench | ASR (%): 6.54 -> 77.88 | Improved |
| add malicious activation-guided selection on top of FGI (ablation) | ActMan on MiniGPT4 · AdvBench | ASR (%): 77.88 -> 79.42 | Improved |
| add malicious activation-guided selection on top of FGI (ablation) | ActMan on LLaVA · AdvBench | ASR (%): 66.92 -> 67.69 | Improved |
| add visual camouflage to full semantic induction module (ablation) | ActMan on MiniGPT4 · AdvBench | ASR (%): 79.42 -> 80.38 | Improved |
| increase PGD perturbation budget epsilon (ablation) | ActMan · AdvBench and MMSafetyBench | ASR (%) | Improved |
| increase number of generated scenarios R (ablation) | ActMan · AdvBench and MMSafetyBench | ASR (%) | Improved |
| use ActMan instead of HIMRD under ASTRA defense | ActMan vs HIMRD · AdvBench (MiniGPT4, ASTRA) | ASR (%): 43.46 -> 54.81 | Improved |
| use ActMan instead of HIMRD | ActMan vs HIMRD · AdvBench (QwenVL, white-box) | ASR (%): 64.42 -> 62.69 | Mixed |
Closest papers by meaning
- Jailbreaking Vision-Language Models Through the Visual Modality (ICML 2026 · similarity 0.89)
- MMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Vision Language Models (AAAI 2025 · similarity 0.89)
- JailBound: Jailbreaking Internal Safety Boundaries of Vision-Language Models (NeurIPS 2025 · similarity 0.89)
- Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreaks (CVPR 2025 · similarity 0.89)
- FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts (AAAI 2025 · similarity 0.88)
- Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
