Research map / Vision-language and multimodal / CLIP and zero-shot

DART: Dual-Modal Adaptive Online Prompting and Knowledge Retention for Test-Time Adaptation

AAAI 2024 · Vision-language and multimodal › CLIP and zero-shot · approach: Prompt learning for CLIP · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

DART adapts CLIP at test time by learning class-specific text prompts and instance-level image prompts per test sample, and retains knowledge from past samples via dual-modal knowledge retention prompts.

Problem

Test-time generalization of CLIP-based vision-language models under training-test distribution shift, where existing test-time prompt tuning uses only text prompts and cannot reuse knowledge from previously seen test samples.

Approach

Extends test-time prompt tuning (TPT) to both modalities with class-specific text prompts and instance-level image prompts, and adds dual-modal knowledge retention prompts that are confidence-weighted and merged into subsequent samples' prompt initialization.

Key results

• DART reaches 63.29% average Acc@1 across ImageNet-A, ImageNet-R and ImageNet-Sketch with CLIP ViT-B/16, versus 59.92% for TPT.
• On ImageNet-A, DART achieves 60.56% Acc@1 vs 54.77% for TPT (+5.79).
• Ablation on ImageNet-A shows the full dual-modal prompting plus knowledge retention reaches 60.56% Acc@1 vs 47.87% for zero-shot CLIP.

Measured effects reported in the paper

Surprising findings

• Learning the CLS token alongside class-specific text prompts at test time hurts performance, per the authors' ablation.

Limitations

• Image prompts are added to only one layer because online learning sees only one test sample, which limits capacity.
• Evaluated only on ImageNet variant benchmarks with a single CLIP backbone family.

Experimental setup

Datasets: ImageNet-A ImageNet-R ImageNet-Sketch

Benchmarks: ImageNet-A ImageNet-R ImageNet-Sketch

Metrics: Acc@1 Acc@5

Baselines: CLIP Ensemble CoOp CoCoOp MaPLe TPT Tent EATA SAR RMT

Builds on: TPT CLIP VPT CoOp MaPLe

Closest papers by meaning

• Dual Prototype Evolving for Test-Time Generalization of Vision-Language Models (NeurIPS 2024 · similarity 0.89)
• Hierarchical Knowledge Prompt Tuning for Multi-task Test-Time Adaptation (CVPR 2025 · similarity 0.88)
• The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.87)
• Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning (NeurIPS 2026 · similarity 0.87)
• DPCore: Dynamic Prompt Coreset for Continual Test-Time Adaptation (ICML 2025 · similarity 0.87)
• PromptKD: Unsupervised Prompt Distillation for Vision-Language Models (CVPR 2024 · similarity 0.87)