Research map / Vision-language and multimodal / CLIP and zero-shot
Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning
Chun-Mei Feng, Kai Yu, Yong Liu, Salman Khan, Wangmeng Zuo
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace random-crop TPT augmentation with diffusion augmentation plus cosine filtration (DiffTPT) | TPT on CLIP-RN50 · ImageNet-A | top-1 accuracy: 27.34 -> 31.06 | Improved |
| apply DiffTPT instead of TPT | CLIP-RN50 OOD average · S1 OOD average | top-1 accuracy: 43.46 -> 45.69 | Improved |
| apply DiffTPT on top of CoOp prompts | CoOp (RN50) · S1 average | top-1 accuracy: 47.12 -> 50.87 | Improved |
| apply TPT on top of CoCoOp with ViT-B/16 | CoCoOp (ViT-B/16) · S1 average | top-1 accuracy: 61.01 -> 60.17 | Hurt |
| apply DiffTPT on top of CoCoOp with ViT-B/16 | CoCoOp (ViT-B/16) · S1 average | top-1 accuracy: 61.01 -> 61.57 | Improved |
| use DiffTPT instead of TPT on cross-dataset generalization | TPT · S2 cross-dataset average | average top-1 accuracy: 55.12 -> 59.85 (RN50); 63.03 -> 65.47 (ViT-B/16) | Improved |
| increase number of augmented views N up to 64 (ablation) | DiffTPT · S1 and S2 | top-1 accuracy | Improved |
| increase N to 128 (ablation) | DiffTPT · S1 and S2 | top-1 accuracy | No clear effect |
| increase prompt update steps from 1 to 4 (ablation) | DiffTPT · S2 | average top-1 accuracy: 56.6 -> 58.8 | Improved |
| increase prompt update steps beyond 5 (ablation) | DiffTPT · S2 | average top-1 accuracy | Hurt |
| raise share of diffusion-based augmentation beyond ratio 0.4 (ablation) | DiffTPT · S1 | top-1 accuracy | Hurt |
| TPT on EuroSAT versus zero-shot CLIP | TPT · EuroSAT | top-1 accuracy: reduces accuracy by 0.87 (RN50) and 3.75 (ViT-B/16) | Hurt |
Closest papers by meaning
- Robust Test-Time Adaptation for Zero-Shot Prompt Tuning (AAAI 2024 · similarity 0.90)
- C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion (ICLR 2024 · similarity 0.90)
- DynaPrompt: Dynamic Test-Time Prompt Tuning (ICLR 2025 · similarity 0.89)
- Black-Box Test-Time Prompt Tuning for Vision-Language Models (AAAI 2025 · similarity 0.89)
- On the Test-Time Zero-Shot Generalization of Vision-Language Models: Do We Really Need Prompt Learning? (CVPR 2024 · similarity 0.87)
- Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
