AI Paper Atlas

Research map / Adversarial robustness and security / Attacks on text and graphs

VEAttack: Downstream-agnostic Vision Encoder Attack against Large Vision Language Models

Hefei Mei, Zirui Wang, Shen You, Minjing Dong, Chang Xu

ICLR 2026 · Adversarial robustness and security › Attacks on text and graphs · approach: Vision-language model robustness

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
attack patch token features instead of class token (ablation)VEAttack · COCO (eps=4/255, LLaVa1.5-7B)CIDEr after attack (lower better): 25.5 -> 7.1Improved
also attack class token alongside patch tokens (ablation)VEAttack · COCO (eps=4/255, LLaVa1.5-7B)CIDEr after attack (lower better): 7.1 -> 10.5Hurt
use cosine similarity loss instead of Euclidean (ablation)VEAttack · COCO (eps=4/255, LLaVa1.5-7B)CIDEr after attack (lower better): 39.6 -> 7.1Improved
use cosine similarity instead of K-L divergence (ablation)VEAttack · VQAv2 (eps=4/255, LLaVa1.5-7B)VQA accuracy after attack (lower better): 33.2 (K-L) vs 38.4 (cosine)Mixed
reduce attack steps from 100 to 10 (ablation)VEAttack · COCO and TextVQAperformance degradation: 84.4% reduction at t=10No clear effect
increase perturbation budget beyond 8/255 (ablation)VEAttack · COCO and TextVQAperformance degradationNo clear effect
use a robust (FARE) vision encoder as the victimLLaVa1.5-7B · COCO transferCIDEr after attack: 3.7 -> 48.6 (FARE self-attack)Improved
craft adversarial examples on a robust encoder (FARE) instead of CLIPVEAttack transfer · COCO transfer to CLIP-based LLaVa1.5-7BCIDEr after attack: CLIP source: 3.7 own, FARE source: 38.7 on CLIP targetImproved
switch from white-box Ensemble to VEAttack (eps=8/255, t=50)attack pipeline · COCO full datasetattack time (h): 41.9 -> 5.3Improved

Later papers that compared with this method

1 later accepted paper reports VEAttack as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy