Research map / Adversarial robustness and security / Attacks on text and graphs
NAP-Tuning: Neural Augmented Prompt Tuning for Adversarially Robust Vision-Language Models
Jiaming Zhang, Xin Wang, Xingjun Ma, Lingyu Qiu, Yu–Gang Jiang, Jitao Sang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TokenRefiner depth of 2 layers (versus none) (ablation) | NAP-Tuning · ImageNet and Flowers, PGD-5 training | accuracy | Improved |
| add Neural Augmentor module (ablation) | NAP-Tuning · ImageNet and Flowers, varying prompt depth | robust accuracy | Improved |
| add image-side and text-side TokenRefiner (dual-modal vs single) (ablation) | NAP-Tuning · 11 datasets, PGD-100 | robust accuracy | Improved |
| increase prompt depth from 1 to 12 layers (ablation) | NAP-Tuning · 11 datasets | clean and robust accuracy | Mixed |
| increase shots from 1 to 16 (ablation) | NAP-Tuning · 11-dataset average | clean and robust accuracy: 40.6 -> 73.5 clean; 14.1 -> 49.9 robust | Improved |
| increase adversarial regularization alpha (ablation) | NAP-Tuning · Flowers | robust accuracy | Improved |
| increase context vector count 1 to 16 (ablation) | NAP-Tuning · ImageNet and Flowers | accuracy | No clear effect |
| raise training eps from 1 to 4 (ablation) | NAP-Tuning · Flowers | robust accuracy at test eps=8: 0.1 -> 11.8 | Improved |
| raise training eps from 1 to 4 (ablation) | NAP-Tuning · Flowers | clean accuracy: 96.1 -> 87.5 | Hurt |
| combine with DiffPure image purification (ablation) | NAP-Tuning · 11 datasets | robust accuracy: 12.9 percentage points average | Improved |
| freeze encoder instead of fine-tuning (NAP-Tuning vs TeCoA) | CLIP ViT-B/32 · PGD-100 | clean accuracy: 36.8 -> 69.1 | Improved |
| use deep multi-layer prompts instead of AdvPT input-level prompt | CLIP ViT-B/16 · PGD-100 average | robust accuracy: 4.4 -> 49.9 | Improved |
Closest papers by meaning
- One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models (CVPR 2024 · similarity 0.92)
- Amend to Alignment: Decoupled Prompt Tuning for Mitigating Spurious Correlation in Vision-Language Models (ICML 2024 · similarity 0.91)
- R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuning (CVPR 2025 · similarity 0.91)
- Learning to Learn Better Visual Prompts (AAAI 2024 · similarity 0.89)
- TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models (CVPR 2025 · similarity 0.89)
- Nemesis: Normalizing the Soft-prompt Vectors of Vision-Language Models (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
