Research map / Vision-language and multimodal / CLIP and zero-shot
GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GS-Bias, a single-image test-time adaptation method that adds learnable global and spatial logit biases to frozen CLIP outputs, avoiding full-network backpropagation and secondary inference.
Problem
Test-time adaptation of VLMs faces a trade-off: prompt optimizers (TPT) are accurate but memory- and compute-heavy, while training-free visual optimizers (MTA) are efficient but unstable in zero-shot and unseen-class settings.
Approach
Instead of tuning text prompts or visual features, learn two zero-initialized bias vectors added directly to CLIP's output logits: a global bias trained by entropy minimization across augmented views, and a spatial bias trained by entropy minimization over top-K class-relevant patch-level predictions from the vision encoder.
Key results
- On 10 cross-dataset benchmarks with ViT-B/16, GS-Bias + E. reaches 67.03% average vs. 64.80% for TPT (BS=64), with GS-Bias alone at 64.86%.
- On ImageNet, GS-Bias + E. uses 1308 MiB memory vs. 19997 MiB for TPT (about 6.5%), with 70.57% accuracy vs. 68.48% for TPT.
- On domain generalization with ViT-B/16, GS-Bias + E. reaches 64.52% average and 63.01% OOD average, above MTA + E. (64.08% / 62.53%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add global and spatial biases to CLIP logits | CLIP ViT-B/16 · Cross-dataset generalization (10 datasets) | top-1 accuracy (Average): 65.64 -> 67.03 | Helped |
| Add global and spatial biases to CLIP logits | CLIP ViT-B/16 · Domain generalization (OOD Average) | top-1 accuracy: 62.53 -> 63.01 | Helped |
| Optimize adaptation biases at the output logits stage | CLIP ViT-B/16 on ImageNet · ImageNet | memory (MiB): 19997 -> 1308 | Helped |
| Add global bias (ablation) | GS-Bias · ImageNet | top-1 accuracy: 68.77 -> 70.45 | Helped |
| Add spatial bias (ablation) | GS-Bias with global bias · ImageNet | top-1 accuracy: 70.45 -> 70.57 | Helped |
| Add spatial bias (ablation) | GS-Bias without global bias · 10 Cross-Datasets | top-1 accuracy: 65.64 -> 65.77 | Helped |
| Add global bias (ablation) | GS-Bias with spatial bias · 10 Cross-Datasets | top-1 accuracy: 65.77 -> 67.03 | Helped |
| Replace learned spatial views with randomly cropped Crop 1×1 views (ablation) | GS-Bias · 10 Cross-Datasets | top-1 accuracy: 67.03 -> 66.02 | Hurt |
| Replace learned spatial views with randomly cropped Crop 2×2 views (ablation) | GS-Bias · 10 Cross-Datasets | top-1 accuracy: 67.03 -> 66.03 | Hurt |
| Add TDA historical data stream | GS-Bias · 10 Cross-Datasets | top-1 accuracy: 67.03 -> 68.30 | Helped |
| Increase global bias learning rate α (ablation) | GS-Bias · 10 Cross-Datasets | top-1 accuracy: 67.03 -> 66.30 -> 66.26 -> 66.11 -> 66.00 | Hurt |
| Increase global bias learning rate α (ablation) | GS-Bias · ImageNet | top-1 accuracy: 70.08 -> 70.51 -> 70.57 -> 70.55 -> 70.54 | Mixed |
Surprising findings
- Reducing crop size for augmented views (1x1 or 2x2 crops) as substitutes for spatial views gives no gain over GS-Bias without spatial bias, suggesting the region-based spatial features matter more than cropping.
Limitations
- Relies on empirically selected hyperparameters (learning rates, K, views selection rate) that may need reconfiguration per test sample.
- Cross-dataset gains over MTA without prompt ensembling are modest; GS-Bias alone scores 64.86 vs MTA 64.43 on the 10-dataset average.
- Performance on cross-dataset benchmarks declines with too many learning steps or large learning rates, indicating overfitting risk.
Experimental setup
Datasets: ImageNet Flowers102 DTD OxfordPets StanfordCars FGVC-Aircraft Food101 EuroSAT UCF101 SUN397 Caltech101
Benchmarks: Flowers102 DTD OxfordPets StanfordCars UCF101 Caltech101 Food101 SUN397 FGVC-Aircraft EuroSAT ImageNet ImageNet-V2
Metrics: top-1 accuracy FPS memory (MiB)
Baselines: CLIP CoOp CoCoOp TPT DiffTPT MTA
Builds on: TPT MTA CLIP DiffTPT
Closest papers by meaning
- The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.90)
- Frustratingly Easy Test-Time Adaptation of Vision-Language Models (NeurIPS 2024 · similarity 0.89)
- BoostAdapter: Improving Vision-Language Test-Time Adaptation via Regional Bootstrapping (NeurIPS 2024 · similarity 0.89)
- DOTA: Distributional Test-time Adaptation of Vision-Language Models (NeurIPS 2025 · similarity 0.88)
- On the Test-Time Zero-Shot Generalization of Vision-Language Models: Do We Really Need Prompt Learning? (CVPR 2024 · similarity 0.88)
- Language Bias in LVLMs: From In-Depth Analysis to Simple and Effective Mitigation (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.