AI Paper Atlas

Research map / Adversarial robustness and security / Attacks on text and graphs

Adversarial Prompt Tuning for Vision-Language Models

Jiaming Zhang, Xingjun Ma, Xin Wang, Lingyu Qiu, Jiaqi Wang, Yu-Gang Jiang, Jitao Sang

ECCV 2024 · Adversarial robustness and security › Attacks on text and graphs · approach: Vision-language model robustness · 38 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
tune learnable text prompts against an adversarial embedding bank (AdvPT) instead of hand-crafted promptsCLIP ViT-B/16 · ImageNet (PGD-40)robust accuracy (%): 6.6 -> 19.9Improved
apply AdvPT against AutoAttackCLIP ViT-B/16 · Flowers (AutoAttack)accuracy (%): 0.0 -> 23.3Improved
combine AdvPT with Super-resolution input denoisingSuper-resolution defense, CLIP ViT-B/16 · Flowers (PGD-40)robust accuracy (%): 13.8 -> 60.4Improved
combine AdvPT with DiffPureDiffPure, CLIP ViT-L/14 · Pets (PGD-40)robust accuracy (%): 90.6 -> 90.3No clear effect
combine AdvPT with DiffPureDiffPure, CLIP ViT-B/16 · Flowers (PGD-40)robust accuracy (%): 59.4 -> 81.9Improved
train prompts with adversarial objective (AdvPT) instead of clean CoOp objective (ablation)prompt tuning on CLIP · multiple datasetsrobustness vs generalizationMixed
use AdvPT instead of adversarial training (AT)CLIP on Pets · Petstraining time: at least 100x more efficientImproved
linear probing on downstream data instead of adversarial prompt tuning (ablation)CLIP · Flowers, Pets and others (PGD-40)robust accuracy (%): 6.4 -> 4.8Hurt
vary number of learnable vectors M from 1 to 50 (ablation)AdvPT · Petsrobust accuracyNo clear effect
scale model from ViT-B/16 to ViT-L/14 (ablation)AdvPT · generalization gap vs CoOpclean accuracyImproved
train prompts on ImageNet and test on ImageNet variantsAdvPT · ImageNetV2, Sketch, A, Rrobust accuracyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy