Research map / Vision-language and multimodal / CLIP and zero-shot
Batch-Conditioned Semantic Anchors for Robust Transductive Adaptation of Vision--Language Models
Mohammed Rahman Sherif Khan Mohammad, Ardhendu Behera, Sandip Pradhan, Swagat Kumar, Amr Ahmed
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes batch-conditioned semantic anchors for transductive test-time adaptation of CLIP-style vision-language models, balancing batch exploitation against preserving text semantics.
Problem
Transductive adaptation of vision-language models on unlabeled test batches faces a trade-off: aggressive methods drift under sparse batches, prior shift, or limited label-space coverage, while conservative methods keep CLIP text semantics but under-adapt.
Approach
Makes the semantic anchors used to refine predictions conditioned on the unlabeled test batch rather than fixed text embeddings, so adaptation can use batch structure while staying anchored to CLIP text semantics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use batch-aware transported anchor instead of fixed CLIP text anchor (ablation) | JPTA · Ablation mean (Aircraft, EuroSAT, Flowers) | top-1 accuracy (%): 50.23 -> 60.90 | Improved |
| blend transported anchor with original text anchor rather than transported-only (ablation) | JPTA · Ablation mean (Aircraft, EuroSAT, Flowers) | top-1 accuracy (%): 57.26 -> 60.90 | Improved |
| add batch-marginal prior-aware refinement (ablation) | JPTA · Ablation mean (Aircraft, EuroSAT, Flowers) | top-1 accuracy (%): 54.66 -> 60.90 | Improved |
| add regime-adaptive trust control (ablation) | JPTA · Ablation mean (Aircraft, EuroSAT, Flowers) | top-1 accuracy (%): 54.50 -> 60.90 | Improved |
| use adaptive pseudocounts for anchored statistics instead of fixed schedule (ablation) | JPTA · Ablation mean (Aircraft, EuroSAT, Flowers) | top-1 accuracy (%): 57.79 -> 60.90 | Improved |
| free transduction (TransCLIP) on sparse low-coverage batches | TransCLIP · B=64 Very Low, 11-dataset avg | top-1 accuracy (%): 65.2 (CLIP) -> 38.9 | Hurt |
| replace fixed anchoring (StatA) with JPTA transported anchor in sparse batches | StatA · B=64 Very Low, 11-dataset avg | top-1 accuracy (%): 70.4 -> 82.4 | Improved |
| replace StatA with JPTA in large high-coverage batches | StatA · B=1000 High (25-50), per dataset | top-1 accuracy (%): 69.7 -> 73.1 avg | Mixed |
| replace StatA with JPTA in all-class evaluation | StatA · All-class, 11-dataset avg | top-1 accuracy (%): 69.9 -> 73.3 | Improved |
| replace StatA with JPTA in online correlated streams | StatA · Online stream Separate | top-1 accuracy (%): 69.1 -> 75.1 | Improved |
| transfer JPTA config unchanged to SigLIP2 backbone | JPTA · SigLIP2 So400m/14-384, 11 datasets x 3 regimes | conditions improved over zero-shot: 29/33 improved | Mixed |
| random hyperparameter perturbations of regularization weights (0.5x-2x) (ablation) | JPTA · Very Low, High, All-Class macro | macro accuracy difference (pp): worst impact -1.33 | Hurt |
Closest papers by meaning
- Frustratingly Easy Test-Time Adaptation of Vision-Language Models (NeurIPS 2024 · similarity 0.90)
- A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models (CVPR 2024 · similarity 0.90)
- Boosting Vision-Language Models with Transduction (NeurIPS 2024 · similarity 0.90)
- The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.90)
- Rethinking Misalignment in Vision-Language Model Adaptation from a Causal Perspective (NeurIPS 2024 · similarity 0.90)
- Realistic Test-Time Adaptation of Vision-Language Models (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
