AI Paper Atlas

Research map / Adversarial robustness and security / Attacks on text and graphs

IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselves

Ruofan Wang, Juncheng Li, Yixu Wang, Bo Wang, Xiaosen Wang, Yan Teng, Yingchun Wang, Xingjun Ma, Yu-Gang Jiang

ICCV 2025 · Adversarial robustness and security › Attacks on text and graphs · approach: LLM jailbreak and injection defense · 36 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use IDEATOR (black-box) instead of no attackMiniGPT-4 · AdvBenchASR (%): 35.0 -> 94.0Improved
use IDEATOR instead of MM-SafetyBenchMiniGPT-4 · AdvBenchASR (%): 66.0 -> 94.0Improved
use IDEATOR instead of MM-SafetyBench for transfer to LLaVAIDEATOR · AdvBench transferASR (%): 46.0 -> 82.0Improved
use IDEATOR instead of MM-SafetyBench for transfer to ChameleonIDEATOR · AdvBench transferASR (%): 22.0 -> 75.0Improved
increase exploration breadth and depth from Nb=1,Nd=1 to Nb=7,Nd=3 (ablation)IDEATOR · AdvBenchASR (%): 45.0 -> 94.0Improved
increase breadth from 1 to 7 at depth 1 (ablation)IDEATOR · AdvBenchASR (%): 45.0 -> 85.0Improved
increase depth from 1 to 3 at breadth 1 (ablation)IDEATOR · AdvBenchASR (%): 45.0 -> 68.0Improved
add text component to image-only attack (Adv I+T vs Adv Img) (ablation)IDEATOR · AdvBenchASR (%): 85.0 -> 94.0Improved
add image component to text-only attack (Adv I+T vs Adv Text) (ablation)IDEATOR · AdvBenchASR (%): 86.0 -> 94.0Improved
apply AdaShield-S defense against IDEATOR on MiniGPT-4IDEATOR · AdvBench (defense)ASR (%): 94.0 -> 84.0Hurt
apply AdaShield-S defense against MM-SafetyBench on LLaVAMM-SafetyBench · AdvBench (defense)ASR (%): 46.0 -> 5.0Hurt
use Gemini-based IDEATOR challenge set vs base set on Claude-3.5-SonnetVLJailbreakBench · VLJailbreakBenchASR (%): 1.09 -> 19.65Improved

Later papers that compared with this method

1 later accepted paper reports IDEATOR as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy