Research map / Vision-language and multimodal / CLIP and zero-shot

What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective

NeurIPS 2026 · Vision-language and multimodal › CLIP and zero-shot · approach: Test-time adaptation of VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TTABC, an open-source benchmark that unifies 20+ test-time adaptation methods for CLIP into parameter-based, state-based, and inference-based paradigms, and runs controlled studies on what drives their gains across natural, fine-grained, corruption, and label shifts.

Problem

Test-time adaptation (TTA) methods for CLIP (TTA4CLIP) have proliferated, but it is unclear what actually drives their gains, where improvements come from, and under which distribution shifts they are reliable.

Approach

Organize TTA4CLIP methods by what is updated at test time (parameters, external state, or nothing—inference only) and use a standardized benchmark with controlled ablations to show gains come mainly from test-time evidence and reliable proxies rather than heavy optimization; no single paradigm wins across shift types.

Key results

• On natural shifts, state-based DoTA reaches 64.16% and parameter-based TPS 64.46% average, versus 61.70% for TPT (reported in text; Table averages differ).
• On ImageNet-C with ViT-B/16, DeYO achieves the best corruption accuracy of 34.98% versus 21.46% for zero-shot CLIP.
• Under extreme separate-class label shift, StatA improves from 46.89% to 53.51%, while most existing TTA4CLIP methods stay near zero-shot.

Surprising findings

• Increasing the number of augmented views gives larger gains than the parameter optimization itself, and TPS saturates after one step.
• Online TPT without episodic reset collapses to around 3% accuracy by the end of the stream.

Limitations

• Scope limited to image classification and CLIP-based architectures.
• Does not cover dense prediction tasks or autoregressive generative VLMs.

Experimental setup

Datasets: ImageNet ImageNet-V2 ImageNet-Sketch ImageNet-A ImageNet-R ImageNet-C StanfordCars Food101 FGVC Aircraft OxfordPets Flowers102 SUN397

Benchmarks: ImageNet-V2 ImageNet-A ImageNet-R ImageNet-Sketch ImageNet-C StanfordCars Food101 FGVC Aircraft OxfordPets Flowers102 SUN397 DTD

Metrics: top-1 accuracy wall-clock time per sample GPU memory

Baselines: TPT DiffTPT HisTPT C-TPT A-TPT R-TPT PromptAlign RLCF TPS DPE BATCLIP TDA

Builds on: TPT Tent DiffTPT TDA DPE

Closest papers by meaning

• The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.93)
• CLIPTTA: Robust Contrastive Vision-Language Test-Time Adaptation (NeurIPS 2025 · similarity 0.91)
• WATT: Weight Average Test Time Adaptation of CLIP (NeurIPS 2024 · similarity 0.90)
• Test-Time Adaptation of Vision-Language Models for Open-Vocabulary Semantic Segmentation (NeurIPS 2025 · similarity 0.90)
• DOTA: Distributional Test-time Adaptation of Vision-Language Models (NeurIPS 2025 · similarity 0.90)
• Efficient Test-time Adaptation through Candidate Verification and Divergence Shifts (NeurIPS 2026 · similarity 0.90)