Research map / Adversarial robustness and security / Attacks on text and graphs
ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behavior
Weikai Lu, Ziqian Zeng, Kehua Zhang, Haoran Li, Huiping Zhuang, Ruidong Wang, Cen Chen, Hao Peng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ARGUS activation-steering defense | Qwen2-VL-7B-Instruct · Multimodal IPI test set (Image) | AIA: 25.1 -> 0.1 | Improved |
| add ARGUS defense | Qwen2-VL-7B-Instruct · Multimodal IPI test set (Image) | UIA (inject): 30.9 -> 46.3 | Improved |
| add optimal utility direction search (ablation) | ARGUS · Multimodal IPI test set (Image) | UIA (inject): 44.5 -> 46.3 | Improved |
| add optimal utility direction search (ablation) | ARGUS · Multimodal IPI test set (Audio) | UIA (inject): 54.4 -> 58.0 | Improved |
| add adaptive intervention strength (ablation) | ARGUS · Multimodal IPI test set (Image) | UIA (inject): 45.9 -> 46.3 | Improved |
| add adaptive intervention strength (ablation) | ARGUS · Multimodal IPI test set (Video) | UIA (inject): 38.0 -> 37.8 | Hurt |
| add post-filtering stage (ablation) | ARGUS · Multimodal IPI test set (Image) | AIA: 4.3 -> 0.1 | Improved |
| add post-filtering stage (ablation) | ARGUS · Multimodal IPI test set (Image) | UIA (inject): 46.4 -> 46.3 | Hurt |
| add Removal (editing model removes injected text) | Qwen2-VL-7B-Instruct · Multimodal IPI test set (Image) | AIA: 25.1 -> 0.0 | Improved |
| add Noise to the modality input | Qwen2-VL-7B-Instruct · Multimodal IPI test set (Video) | UIA (clean): 37.6 -> 23.3 | Hurt |
| add System Prompt defensive instruction | Qwen2-VL-7B-Instruct · Multimodal IPI test set (Video) | AIA: 28.2 -> 26.9 | No clear effect |
| add Ignore Prompt counter-instruction | Qwen2-VL-7B-Instruct · Multimodal IPI test set (Image) | AIA: 25.1 -> 31.5 | Hurt |
Closest papers by meaning
- ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents (ICLR 2026 · similarity 0.87)
- Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models? (CVPR 2025 · similarity 0.87)
- Dictionary-Aligned Concept Control for Safeguarding Multimodal LLMs (CVPR 2026 · similarity 0.87)
- Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy (ICLR 2025 · similarity 0.86)
- ALMGuard: Safety Shortcuts and Where to Find Them as Guardrails for Audio–Language Models (NeurIPS 2025 · similarity 0.86)
- Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
