Research map / Label-efficient and robust learning / Test-time adaptation
BoostAdapter: Improving Vision-Language Test-Time Adaptation via Regional Bootstrapping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BoostAdapter, a training-free test-time adapter for CLIP that augments a key-value cache with historical test samples plus instance-specific boosting samples from augmented views of each test image, with theoretical error bounds.
Problem
Test-time adaptation of vision-language models like CLIP: training-required methods (TPT) are computationally expensive, while training-free cache methods (TDA) ignore information within the test sample itself.
Approach
Shows that entropy minimization over low-entropy augmented views (boosting samples) behaves like a cache classifier over those samples, and uses a key-value cache that combines instance-agnostic historical samples with instance-aware boosting samples from regional cropping of each test image, with no gradient updates.
Key results
- Average OOD accuracy with ViT-B/16 of 65.57% vs 63.89% for TDA and 60.81% for CLIP+TPT.
- Cross-domain average of 68.68% with ViT-B/16 vs 67.53% for TDA, improving on 7 of 10 datasets.
- Inference speed of 11.23 fps vs 0.29 fps for TPT on a single 3090 GPU, with 1.2 GB memory, matching TDA's cost.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add regional-bootstrapped boosting samples to historical samples (ablation) | BoostAdapter · OOD benchmark (ViT-B/16), Average | top-1 accuracy: 64.85 -> 65.57 | Helped |
| use historical samples and boosting samples together rather than boosting samples only (ablation) | BoostAdapter · OOD benchmark (ViT-B/16), Average | top-1 accuracy: 65.34 -> 65.57 | Helped |
| increase the number of augmented views used to generate boosting samples (ablation) | BoostAdapter · UCF101 | top-1 accuracy | Helped |
| increase total cache shot capacity (ablation) | BoostAdapter · Aircraft | top-1 accuracy | Mixed |
| use feature retrieval instead of entropy minimization (ablation) | test-time adaptation methods · Flower102 | top-1 accuracy | Helped |
| add the TSD technique (ablation) | BoostAdapter · OOD benchmark (ViT-B/16), Average | top-1 accuracy: 65.57 -> 65.63 | Helped |
| add the DEYO technique (ablation) | BoostAdapter · OOD benchmark (ViT-B/16), Average | top-1 accuracy: 65.57 -> 65.83 | Helped |
| use BoostAdapter instead of TDA | CLIP ViT-B/16 · OOD benchmark, Average | top-1 accuracy: 63.89 -> 65.57 | Helped |
| use BoostAdapter instead of TDA | CLIP ViT-B/16 · Cross-Domain benchmark, Average | top-1 accuracy: 67.53 -> 68.68 | Helped |
| use BoostAdapter instead of TDA | CLIP RN-50 · OOD benchmark, Average | top-1 accuracy: 46.63 -> 48.20 | Helped |
| use BoostAdapter instead of TDA | CLIP RN-50 · Cross-Domain benchmark, Average | top-1 accuracy: 61.03 -> 61.54 | Helped |
| use BoostAdapter instead of TDA | CLIP ViT-B/16 · ImageNet-C severity 5, Average | accuracy (%): 31.58 -> 32.17 | Helped |
Surprising findings
- Entropy-minimization-based TPT underperforms cache-based TDA-style methods on the OOD benchmark when both use the same number of views, per the paper's adaptation-method ablation.
Limitations
- Requires 64 augmented views per test sample, adding overhead relative to pure training-free adapters.
- Uses random cropping as augmentation, which the authors note is not an efficient or optimal way to obtain boosting samples.
- Theoretical bounds rely on strong density, Lipschitz, and low-noise assumptions and are stated informally in parts.
Experimental setup
Datasets: ImageNet-V2 ImageNet-Sketch ImageNet-A ImageNet-R Caltech101 Pets Cars Flowers102 Food101 Aircraft SUN397 DTD
Benchmarks: ImageNet-V2 ImageNet-Sketch ImageNet-A ImageNet-R Caltech101 Pets Cars Flowers102 Food101 Aircraft SUN397 DTD
Metrics: top-1 accuracy fps memory (GB)
Baselines: CLIP TPT CoOp CoCoOp MaPLe PromptAlign DiffTPT TDA CALIP CuPL
Builds on: TPT TDA Tip-Adapter CLIP
Closest papers by meaning
- Efficient Test-Time Adaptation of Vision-Language Models (CVPR 2024 · similarity 0.92)
- The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.92)
- DOTA: Distributional Test-time Adaptation of Vision-Language Models (NeurIPS 2025 · similarity 0.92)
- Efficient Test-time Adaptation through Candidate Verification and Divergence Shifts (NeurIPS 2026 · similarity 0.91)
- Frustratingly Easy Test-Time Adaptation of Vision-Language Models (NeurIPS 2024 · similarity 0.91)
- Test-Time Adaptation of Vision-Language Models for Open-Vocabulary Semantic Segmentation (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.