Research map / Adversarial robustness and security / Attacks on text and graphs
Localize and Neutralize: Gradient-Guided Token Suppression Against Visual Prompt Injection Attack
Dongpeng Zhang, Ke Ma, Yangbangyan Jiang, Gaozheng Pei, Longtao Huang, Qianqian Xu, Qingming Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use hidden-state gradient norm instead of output logit gradient for token attribution (ablation) | GTM · jailbreak, aligned first token | attack success rate (%): 30.91 -> 24.55 | Improved |
| use hidden-state gradient norm instead of logit gradient (ablation) | GTM · misaligned first token | attack success rate (%): 0.91 -> 0.91 | No clear effect |
| raise masked token ratio beyond 5% (ablation) | GTM · ASR and MM-Vet | ASR and utility | Mixed |
| apply GTM defense | Qwen2-VL-7B · VMA attacks | average ASR (%): 90.54 -> 0.00 | Improved |
| apply GTM defense | LLaVA-v1.5-7b · VMA attacks | average ASR (%): 99.64 -> 1.27 | Improved |
| use GTM instead of DiffPure | defense · MM-Vet utility and cost | utility, time, memory: 7.22 s -> 3.50 s | Improved |
| use GTM instead of Cider under unbounded perturbation | jailbreak defense · jailbreak eps=inf | attack success rate (%): 94.55 -> 0.91 | Improved |
| apply GTM against optimization-based attacks APGD | GTM · APGD eps=32 | attack success rate: 6 vs Guard 6 | No clear effect |
Closest papers by meaning
- Scaling Laws for Adversarial Attacks on Language Model Activations and Tokens (ICLR 2025 · similarity 0.87)
- Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Models (CVPR 2025 · similarity 0.87)
- Boosting the Transferability of Adversarial Attack on Vision Transformer with Adaptive Token Tuning (NeurIPS 2024 · similarity 0.86)
- Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models (ICLR 2024 · similarity 0.86)
- Fight Back Against Jailbreaking via Prompt Adversarial Tuning (NeurIPS 2024 · similarity 0.86)
- VERA-V: Variational Inference Framework for Jailbreaking Vision-Language Models (ICML 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
