Research map / Label-efficient and robust learning / Test-time adaptation
Training-Free Active Test-Time Adaptation for Vision-Language Models
Jihwan Bang, Sumyeong Ahn, Hwanjun Song, Jae-Gil Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a training-free active test-time adaptation method for vision-language models that selectively queries external labels for hard shifted samples instead of relying solely on self-generated pseudo-labels.
Problem
Test-time adaptation for VLMs is typically fully unsupervised, relying on the model's own pseudo-labels, confidence, or entropy, which is unreliable on hard shifted samples; supervised online adaptation assumes labels for all samples, which is impractical.
Approach
Introduces active, training-free test-time adaptation in which a small budget of external label queries is spent on the most uncertain shifted samples, sitting between fully unsupervised TTA and fully supervised online adaptation. (Details inferred from the abstract framing only.)
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ACTOR retrieval-based correction with queried labels to TDA | TDA · Cross-domain, ViT-B/16 | average accuracy: 68.14 -> 72.66 | Improved |
| add ACTOR to TDA | TDA · Cross-domain, ResNet-50 | average accuracy: 60.78 -> 66.16 | Improved |
| add ACTOR to TDA | TDA · OOD benchmark (5 datasets) | average accuracy: 65.22 -> 66.40 (ViT-B/16); 49.64 -> 50.80 (RN50) | Improved |
| add ACTOR to TDA on ImageNet-V2 | TDA RN50 · ImageNet-V2 | top-1 accuracy: 55.10 -> 54.99 | No clear effect |
| use training-free ACTOR instead of supervised Online-CoOp / Replay-CoOp with same queried labels | CLIP ViT-B/16 · 15 datasets | average accuracy: 65.15 / 65.20 -> 67.65 | Improved |
| add Verifier consistency check, full consistency-and-reliability rule vs 5-NN majority vote (ablation) | ACTOR retrieval correction · 5-dataset subset, TDA ViT-B/16 | accuracy: 67.22 -> 69.50 | Improved |
| add reliability threshold on top of consistency (ablation) | ACTOR retrieval correction · 5-dataset subset | harmful-correction rate: 11.99 -> 5.28 | Improved |
| add consistency check on top of reliability-only reuse (ablation) | ACTOR retrieval correction · 5-dataset subset | accuracy: 67.30 -> 69.50 | Improved |
| increase number of Verifiers from 0 to 1 (ablation) | ACTOR · retrieval correction accuracy | correction accuracy | Improved |
| increase number of Verifiers from 1 to 2 (ablation) | ACTOR · overall accuracy | accuracy | Hurt |
| increase label latency from 1 to 1000 samples (ablation) | ACTOR · DTD | accuracy: 46 -> 42 | Hurt |
| increase label latency from 1 to 1000 samples (ablation) | TDA+ACTOR · ImageNet-S | accuracy: around 43 | No clear effect |
Closest papers by meaning
- The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.95)
- Frustratingly Easy Test-Time Adaptation of Vision-Language Models (NeurIPS 2024 · similarity 0.94)
- Efficient Test-Time Adaptation of Vision-Language Models (CVPR 2024 · similarity 0.94)
- Realistic Test-Time Adaptation of Vision-Language Models (CVPR 2025 · similarity 0.94)
- Efficient Test-time Adaptation through Candidate Verification and Divergence Shifts (NeurIPS 2026 · similarity 0.94)
- DOTA: Distributional Test-time Adaptation of Vision-Language Models (NeurIPS 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
