Research map / Adversarial robustness and security / Attacks on text and graphs
Discrete Latent Features Ablate Adversarial Attack: A Robust Prompt Tuning Framework for VLMs
Yang Chen, Yanbin Wei, James Kwok, Yu Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PerturbShield and logits fusion to CLIP (ablation) | CLIP · EuroSAT 16-shot, eps=4/255 | harmonic mean H: 14.22 -> 24.13 | Improved |
| add PerturbShield and logits fusion to adversarial prompt tuning (ablation) | DEFEAT · EuroSAT 16-shot, eps=4/255 | harmonic mean H: 36.31 -> 56.69 | Improved |
| add PerturbShield without logits fusion (ablation) | DEFEAT · EuroSAT 16-shot, eps=4/255 | harmonic mean H: 36.31 -> 19.56 | Hurt |
| add PerturbShield without logits fusion (ablation) | CLIP · EuroSAT 16-shot, eps=4/255 | harmonic mean H: 14.22 -> 14.99 | Improved |
| add logits fusion on top of PerturbShield (ablation) | DEFEAT · EuroSAT 16-shot, eps=4/255 | harmonic mean H: 19.56 -> 56.69 | Improved |
| reconstruct latent features with VQ-VAE instead of pixels (ablation) | CLIP · EuroSAT 16-shot, eps=4/255 | accuracy/robustness: Acc 9.26, Rob 0% -> Acc 17.90, Rob 12.90 | Improved |
| apply prompt alignment regularization (mu > 0) (ablation) | DEFEAT · Average over datasets | robust accuracy: best at mu = 20 | Improved |
| increase shots from 4 to 16 with DEFEAT vs FAP | DEFEAT · 11-dataset avg, eps=4/255 | robust accuracy: DEFEAT 36.05 vs APT 20.26 and FAP 19.82 | Improved |
| increase training samples from 16 to 100 shots | DEFEAT · ImageNet source | robust accuracy: 19.06 -> 22.71 | Improved |
| adopt DEFEAT over FAP in few-shot accuracy | DEFEAT · 11-dataset avg, eps=1/255, 16-shot | clean accuracy: 66.40 vs 65.03 | No clear effect |
Closest papers by meaning
- One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models (CVPR 2024 · similarity 0.91)
- R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuning (CVPR 2025 · similarity 0.89)
- Amend to Alignment: Decoupled Prompt Tuning for Mitigating Spurious Correlation in Vision-Language Models (ICML 2024 · similarity 0.87)
- TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models (CVPR 2025 · similarity 0.87)
- Improving Zero-Shot Adversarial Robustness in Vision-Language Models by Closed-form Alignment of Adversarial Path Simplices (ICML 2025 · similarity 0.86)
- Few-Shot Adversarial Prompt Learning on Vision-Language Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
