Research map / Vision-language and multimodal / CLIP and zero-shot
Robustifying Vision-Language Models via Test-Time Prompt Adaptation
Xingyu Zhu, Huanshen Wu, Shuo Wang, Beier Zhu, Jiannan Ge, Jiaheng Zhang, Long Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RITA optimal-transport distribution alignment instead of R-TPT | CLIP ViT-B/16 · 8 fine-grained datasets, eps=1.0 | average robust accuracy: 50.5 -> 52.7 | Improved |
| use RITA instead of R-TPT | CLIP ViT-B/32 · 8 fine-grained datasets, eps=1.0 | average robust accuracy: 47.9 -> 49.7 | Improved |
| use RITA instead of R-TPT on Cars | CLIP ViT-B/32 · Cars, eps=1.0 | robust accuracy: 44.6 -> 42.9 | Hurt |
| plug RITA onto adversarially finetuned FARE CLIP | FARE · 8 fine-grained datasets avg | robust accuracy: 23.3 -> 41.0 | Improved |
| use RITA on top of PMG | PMG CLIP · 8 fine-grained datasets avg | robust accuracy: 26.1 -> 32.6 | Improved |
| use RITA against CW attack | CLIP ViT-B/16 · 8 fine-grained avg, CW | robust accuracy: 51.2 -> 54.0 | Improved |
| use RITA against DI attack | CLIP ViT-B/16 · 8 fine-grained avg, DI | robust accuracy: 44.2 -> 46.2 | Improved |
| add dynamic cache term (alpha 0 to 0.1) (ablation) | RITA · DTD and ImageNet, ViT-B/16 | accuracy | Improved |
| increase number of learnable prompts from 4 to 32 (ablation) | RITA · DTD, ViT-B/16 | clean and robust accuracy | Improved |
| increase augmented views from 96 to 128 (ablation) | RITA · DTD, ViT-B/16 | robust accuracy | Hurt |
| increase TTA steps from 1 to 2 (ablation) | RITA · fine-grained datasets and ImageNet | robust accuracy: +0.5 to +0.6 | Improved |
| increase TTA steps from 2 to 4 (ablation) | RITA · fine-grained datasets and ImageNet | robust accuracy | Hurt |
Closest papers by meaning
- One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models (CVPR 2024 · similarity 0.92)
- R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuning (CVPR 2025 · similarity 0.91)
- TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models (CVPR 2025 · similarity 0.91)
- On the Test-Time Zero-Shot Generalization of Vision-Language Models: Do We Really Need Prompt Learning? (CVPR 2024 · similarity 0.90)
- CLIPTTA: Robust Contrastive Vision-Language Test-Time Adaptation (NeurIPS 2025 · similarity 0.90)
- Frustratingly Easy Test-Time Adaptation of Vision-Language Models (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
