Research map / Label-efficient and robust learning / Test-time adaptation
WATT: Weight Average Test Time Adaptation of CLIP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes WATT, a test-time adaptation method for CLIP that adapts LayerNorm parameters with a transductive cross-entropy loss over multiple text templates and averages the adapted weights and text embeddings.
Problem
CLIP's zero-shot image classification degrades under domain shifts, and test-time adaptation of VLMs is needed that works with small batches and preserves zero-shot ability.
Approach
Adapt CLIP's visual LayerNorm parameters using pseudo-labels from multiple text templates, then average the weights across template-specific adapted models (parallel or sequential) and average text embeddings over templates at inference, with no extra trainable modules.
Key results
- WATT-S reaches 73.82% on CIFAR-10-C vs 59.22% for CLIP (ViT-B/32), and 45.57% mean on CIFAR-100-C vs 29.43% for CLIP.
- On CIFAR-10 with ViT-B/32, WATT-P achieves 91.41% vs CLIP 88.74% and TENT 91.69%, so TENT is best there.
- Averaging weights outperforms text averaging and output averaging on CIFAR-100-C (45.57% vs 41.37% text avg. and 42.68% output avg.).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Parallel MTWA | CLIP ViT-B/32 · CIFAR-10.1 | accuracy: 83.25 -> 87.78 | Helped |
| Apply Sequential MTWA | CLIP ViT-B/32 · CIFAR-10-C | accuracy: 59.22 -> 73.82 | Helped |
| Apply Sequential MTWA | CLIP ViT-B/32 · CIFAR-100-C mean | accuracy: 29.43 -> 45.57 | Helped |
| Apply WATT-S instead of WATT-P | WATT on CLIP ViT-B/32 · CIFAR-10 | accuracy: 91.41 -> 91.05 | Hurt |
| Average all template text embeddings at test time instead of using one template (ablation) | WATT-S · CIFAR-10 | accuracy: 90.87\,\scriptsize$\pm$0.10 -> 91.08\,\scriptsize$\pm$0.06 | Helped |
| Average all template text embeddings at test time instead of using one template (ablation) | WATT-S · CIFAR-10-C mean | accuracy: 72.08 -> 72.66 | Helped |
| Use weight averaging rather than text-embedding averaging (ablation) | WATT · CIFAR-100-C mean | accuracy: 41.37 -> 45.57 | Helped |
| Use weight averaging rather than output averaging (ablation) | WATT · CIFAR-100-C mean | accuracy: 42.68 -> 45.57 | Helped |
| Use 1 adaptation iteration per averaging cycle for 10 cycles instead of 2 iterations for 5 cycles (ablation) | WATT-S · CIFAR-10 | accuracy: 91.05\,\scriptsize$\pm$0.06 -> 91.39\,\scriptsize$\pm$0.14 | Helped |
| Use 2 adaptation iterations per averaging cycle for 5 cycles instead of 1 iteration for 10 cycles (ablation) | WATT-S · CIFAR-100-C mean | accuracy: 45.32 -> 45.57 | Helped |
| Increase test batch size (ablation) | WATT-S · CIFAR-10-C | accuracy: 61.26 -> 73.82 | Helped |
| Use cross-entropy with pseudo-labels instead of entropy minimization (ablation) | WATT · CIFAR-10 Gaussian Noise | accuracy: 47.08 -> 63.84 | Helped |
Surprising findings
- Entropy minimization (TENT) collapses on Glass blur in CIFAR-100-C (3.71% accuracy), while cross-entropy pseudo-labeling is stable.
- TENT outperforms WATT on the clean CIFAR-10 split with ViT-B/32 and on several ViT-L/14 splits.
Limitations
- Gains on clean or natural-shift datasets are smaller and not consistently best; TENT and CLIPArTT sometimes win.
- Evaluated only on image classification with CLIP-style backbones; extension to segmentation or detection is left to future work.
- Parallel and sequential variants require multiple adapted model copies per step, adding compute relative to single-model TTA.
Experimental setup
Datasets: CIFAR-10 CIFAR-10.1 CIFAR-10-C CIFAR-100 CIFAR-100-C VisDA-C OfficeHome PACS VLCS
Benchmarks: CIFAR-10 CIFAR-10.1 CIFAR-10-C CIFAR-100 CIFAR-100-C VisDA-C OfficeHome PACS VLCS
Metrics: accuracy
Baselines: CLIP TENT TPT CLIPArTT
Builds on: CLIPArTT Lookaround SWAD TENT TPT
Closest papers by meaning
- What Drives Test-Time Adaptation for CLIP? A Controlled Empirical Study from an Update Perspective (NeurIPS 2026 · similarity 0.90)
- Adaptive Parameter Selection for Tuning Vision-Language Models (CVPR 2025 · similarity 0.88)
- The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.88)
- Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models (ICLR 2024 · similarity 0.88)
- Realistic Test-Time Adaptation of Vision-Language Models (CVPR 2025 · similarity 0.88)
- CLIPTTA: Robust Contrastive Vision-Language Test-Time Adaptation (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.