Research map / Vision-language and multimodal / CLIP and zero-shot
Towards Fine-Grained Robustness: Attention-Guided Test-Time Prompt Tuning for Vision-Language Models
Jia-Wei Hai, Yijun Wang, Xiu-Shen Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add attention refinement, augmentation and TV ensemble together vs TPT-Ensemble | A-TPT · 8 fine-grained datasets, ViT-B/16, adversarial | adversarial accuracy: 31.8 -> 45.7 | Improved |
| add token-gradient attention refinement (A-Refine) alone (ablation) | A-TPT · 8 fine-grained datasets, ViT-B/16 | adversarial and clean accuracy: 31.8 -> 31.8 | No clear effect |
| add attention-guided augmentation and TV ensemble without A-Refine (ablation) | A-TPT · 8 fine-grained datasets, ViT-B/16, adversarial | adversarial accuracy: 31.8 -> 32.3 | No clear effect |
| add TV-based ensemble on top of A-Refine and A-Aug (ablation) | A-TPT · 8 fine-grained datasets, ViT-B/16, adversarial | adversarial accuracy: 41.6 -> 45.7 | Improved |
| add attention-guided augmentation on top of A-Refine and A-TV (ablation) | A-TPT · 8 fine-grained datasets, ViT-B/16, adversarial | adversarial accuracy: 38.1 -> 45.7 | Improved |
| reduce number of views from 64 to 16 (ablation) | A-TPT · Pets, ViT-B/16, adversarial | accuracy and time per sample: 70.5 -> 69.8; 2.71 s -> 0.53 s | Mixed |
| use A-TPT instead of R-TPT | A-TPT · 8 fine-grained datasets ViT-B/32 adversarial | adversarial accuracy: 35.3 -> 40.3 | Improved |
| use A-TPT instead of training-time defenses | A-TPT · 8 fine-grained, ViT-B/32 | clean and adversarial accuracy: adv 40.3 vs FARE 40.3; clean 58.6 vs 42.2 | Improved |
| A-TPT under CW, DeepFool, FGSM attacks vs R-TPT | A-TPT · Flower, DTD, ViT-B/32 | adversarial accuracy: Flower 51.8 -> 55.4; DTD 34.2 -> 39.9 | Improved |
| A-TPT on Aircraft clean accuracy vs CLIP | A-TPT · Aircraft, ViT-B/32 clean | accuracy: 18.1 -> 17.5 | Hurt |
Closest papers by meaning
- R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuning (CVPR 2025 · similarity 0.93)
- One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models (CVPR 2024 · similarity 0.91)
- Black-Box Test-Time Prompt Tuning for Vision-Language Models (AAAI 2025 · similarity 0.91)
- TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models (CVPR 2025 · similarity 0.91)
- Frustratingly Easy Test-Time Adaptation of Vision-Language Models (NeurIPS 2024 · similarity 0.90)
- C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
