Research map / Vision-language and multimodal / Multimodal LLMs
Open Your Eyes: Benchmarking the Detection of Fabricated Realities and Weaponized Ethics in VLMs
Amit Pandey, Aditya Mohan, Phani Sankar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| present the artifact as an image instead of identical text | GPT-5.2 · M-IPI technical attack detection | recall: 98.0 -> 91.3 (∆Recall –6.7) | Hurt |
| present the artifact as an image instead of identical text | GPT-4.1 · M-IPI technical attack detection | recall: 79.9 -> 52.9 (∆Recall –27.0) | Hurt |
| present the artifact as an image instead of identical text | Gemma-3-12B · M-IPI technical attack detection | recall: 88.2 -> 76.2 (∆Recall –12.0) | Hurt |
| present the artifact as an image instead of identical text | Qwen3-VL-8B · M-IPI technical attack detection | recall: 20.7 -> 2.4 (∆Recall –18.3) | Hurt |
| present the artifact as an image instead of identical text | Idefics3-8B · M-IPI technical attack detection | recall: 53.6 -> 35.2 (∆Recall –18.4) | Hurt |
| present the artifact as an image instead of identical text | GPT-5.2 · M-IPI ethics-framed attack detection | recall: 85.0 -> 78.4 (∆Recall –6.6) | Hurt |
| present the artifact as an image instead of identical text | GPT-4.1 · M-IPI ethics-framed attack detection | recall: 73.4 -> 54.7 (∆Recall –18.7) | Hurt |
| present the artifact as an image instead of identical text | Gemma-3-12B · M-IPI ethics-framed attack detection | recall: 63.1 -> 38.1 (∆Recall –25.0) | Hurt |
| present the artifact as an image instead of identical text | Qwen3-VL-8B · M-IPI ethics-framed attack detection | recall: 41.6 -> 24.7 (∆Recall –16.9) | Hurt |
| present the artifact as an image instead of identical text | Idefics3-8B · M-IPI ethics-framed attack detection | recall: 1.6 -> 0.0 (∆Recall –1.6) | Hurt |
| present the artifact as an image instead of text for the agent | GPT-4.1 troubleshooting agent · M-IPI technical attack success | attack success rate: 50.9% -> 48.1% (McNemar p = 0.066) | No clear effect |
| use subtle policy-framed injections instead of direct injections (image input) (ablation) | GPT-4.1 · M-IPI ethics-framed attack detection | recall: 83% -> 40% (VL); 89% -> 63% (LM) | Hurt |
Closest papers by meaning
- Jailbreaking Vision-Language Models Through the Visual Modality (ICML 2026 · similarity 0.88)
- [Re] Boosting the Visual Interpretability of CLIP via Adversarial Fine-Tuning (NeurIPS 2026 · similarity 0.87)
- Seeing Is Not Screening: Multimodal Hidden Instruction Attacks on Agent Skill Scanners (NeurIPS 2026 · similarity 0.87)
- XFeat Revisited: Reproducibility and Evaluation of a Lightweight Image Matcher (NeurIPS 2026 · similarity 0.87)
- Revisiting "Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing" (NeurIPS 2026 · similarity 0.86)
- Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
