Research map / Vision-language and multimodal / CLIP and zero-shot
DynaPrompt: Dynamic Test-Time Prompt Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DynaPrompt, an online prompt buffer for test-time prompt tuning of CLIP that adaptively selects, appends, and deletes prompts via entropy and probability-difference criteria to exploit test-sample relatedness while limiting error accumulation.
Problem
Test-time prompt tuning for vision-language models treats each test sample independently and ignores relatedness among test samples, while naive online tuning suffers prompt collapse from error accumulation under entropy minimization.
Approach
A dynamic online prompt buffer where each test sample selects a subset of prompts whose prediction entropy is lower and whose augmentation-induced probability difference is higher than the initial prompt's (thresholds from v0); if none qualify, a fresh v0 prompt is appended and the oldest inactive prompt is deleted when the buffer is full.
Key results
- On domain generalization, DynaPrompt with hand-crafted prompt reaches 61.81 OoD mean vs. TPT 60.81, and on ImageNet-A 56.17 vs. TPT 54.77.
- Combined with MaPLe, DynaPrompt achieves 64.22 OoD mean vs. MaPLe+TPT 62.31 and PromptAlign 63.56.
- Online TPT collapses to 6.96% on ImageNet-A, while DynaPrompt reaches 56.17%; removing dynamic appending drops ImageNet-based mean to 32.63.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add entropy-based prompt selection (ablation) | DynaPrompt · ImageNet-V2 | accuracy: 61.45 -> 64.67 | Helped |
| Add probability-difference prompt selection (ablation) | DynaPrompt · ImageNet-V2 | accuracy: 61.93 -> 64.67 | Helped |
| Add dynamic prompt appending and deletion (ablation) | DynaPrompt · ImageNet-A | accuracy: 30.74 -> 56.17 | Helped |
| Use long prompt A instead of the original short prompt (ablation) | Online TPT · ImageNet-A | accuracy: 6.96 -> 2.06 | Hurt |
| Use long prompt B instead of the original short prompt (ablation) | Online TPT · ImageNet-A | accuracy: 6.96 -> 4.24 | Hurt |
| Replace sequential single-prompt updates with 10 identically initialized prompts and dynamic selection (ablation) | Online TPT · ImageNet-A | accuracy: 6.96 -> 48.14 | Helped |
| Apply DynaPrompt to MaPLe pretrained prompts | MaPLe · ImageNet-A | accuracy: 58.08 -> 60.72 | Helped |
| Apply DynaPrompt to MaPLe pretrained prompts | MaPLe · DTD | accuracy: 45.87 -> 48.75 | Helped |
Surprising findings
- Naive online test-time prompt tuning collapses to 6.96% accuracy on ImageNet-A, far below the 47.87% zero-shot CLIP baseline.
- Longer trainable prompts (40 tokens) worsen online prompt collapse and even reduce TPT accuracy.
Limitations
- Higher test-time cost than TPT, growing with prompt buffer size.
- Performance varies with test sample order on ImageNet-A and ImageNet-R.
- Gains on cross-dataset fine-grained tasks (e.g., Aircraft, Food101, EuroSAT) are modest.
Experimental setup
Datasets: ImageNet ImageNet-V2 ImageNet-Sketch ImageNet-A ImageNet-R Caltech101 OxfordPets StanfordCars Flowers102 Food101 FGVCAircraft SUN397
Benchmarks: ImageNet ImageNet-V2 ImageNet-Sketch ImageNet-A ImageNet-R Caltech101 OxfordPets StanfordCars Flowers102 Food101 FGVCAircraft SUN397
Metrics: top-1 accuracy
Baselines: CLIP CoOp CoCoOp KgCoOp MaPLe CoPrompt Any-shift Prompt TPT AdaPrompt DiffTPT C-TPT PromptAlign
Builds on: test-time prompt tuning (TPT) CoOp MaPLe AugMix CLIP
Closest papers by meaning
- Black-Box Test-Time Prompt Tuning for Vision-Language Models (AAAI 2025 · similarity 0.91)
- Robust Test-Time Adaptation for Zero-Shot Prompt Tuning (AAAI 2024 · similarity 0.90)
- C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion (ICLR 2024 · similarity 0.89)
- On the Test-Time Zero-Shot Generalization of Vision-Language Models: Do We Really Need Prompt Learning? (CVPR 2024 · similarity 0.89)
- Historical Test-time Prompt Tuning for Vision Foundation Models (NeurIPS 2024 · similarity 0.88)
- Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.