Research map / Label-efficient and robust learning / Test-time adaptation
Multi-Label Test-Time Adaptation with Bound Entropy Minimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ML-TTA with Bound Entropy Minimization, which binds the top-k predicted labels (k set by retrieved captions' textual labels) and adapts CLIP's view and caption prompts per test instance for multi-label test-time adaptation.
Problem
Vanilla entropy-minimization TTA methods increase only the top-1 class probability, which undermines other positive labels in multi-label images, and the number of positive labels per image is unknown at test time.
Approach
Bound Entropy Minimization treats the top-k predicted logits as equal so the entropy loss raises all top-k class probabilities; k is estimated from a retrieved paired caption's noun labels (strong/weak label sets), and captions are treated as pseudo-images with known labels to jointly learn view and caption prompts.
Key results
- ML-TTA with RN50 reaches 60.15 average mAP versus 57.31 for CLIP and 57.69 for TPT across five multi-label benchmarks.
- With ViT-B/16, ML-TTA reaches 64.80 average mAP versus 62.61 for CLIP, improving COCO2014 from 54.42 to 57.52.
- Ablation on RN50 COCO2014 shows BEM adds about 0.36 to 0.5 mAP over VP, CP, and VP+CP baselines, and BEM beats BCE loss by 3-4 mAP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add BEM to the view prompt (ablation) | ML-TTA with view prompt · COCO2014 | mAP: 48.51 -> 48.96; 53.32 -> 53.58 | Helped |
| add BEM to the caption prompt (ablation) | ML-TTA with caption prompt · COCO2014 | mAP: 49.12 -> 49.54; 55.14 -> 55.64 | Helped |
| add BEM to view and caption prompts (ablation) | ML-TTA · COCO2014 | mAP: 51.22 -> 51.58; 57.14 -> 57.52 | Helped |
| use caption prompts instead of view prompts (ablation) | ML-TTA without BEM · COCO2014 | mAP: 48.51 -> 49.12; 53.32 -> 55.14 | Helped |
| combine caption prompts with view prompts (ablation) | ML-TTA without BEM · COCO2014 | mAP: 48.51 -> 51.22; 53.32 -> 57.14 | Helped |
| adapt with ML-TTA rather than the strongest reported baseline, TDA | CLIP RN50 · NUSWIDE | mAP: 42.34 -> 42.53 | Helped |
| adapt with ML-TTA rather than the strongest reported baseline, TDA | CLIP ViT-B/16 · NUSWIDE | mAP: 46.72 -> 46.55 | Hurt |
Surprising findings
- Episodic TTA methods TPT and DiffTPT degrade performance below CLIP on VOC2012 and with ViT-B/16 on several datasets.
- On NUS-WIDE, most TTA methods improve CLIP by only about 1 mAP, which the authors attribute to low image resolution limiting augmentation effectiveness.
Limitations
- Relies on retrieved captions whose noun labels may be noisy or irrelevant, degrading performance with too many captions on VOC2007.
- Performance gains on low-resolution NUS-WIDE are marginal because random augmentation loses small-object information.
- Adds adaptation time relative to TPT due to optimizing two prompts.
Experimental setup
Datasets: MSCOCO VOC NUS-WIDE LLM-generated text description base (PVP-style)
Benchmarks: MSCOCO2014 MSCOCO2017 VOC2007 VOC2012 NUS-WIDE
Metrics: mAP
Baselines: CLIP TPT DiffTPT RLCF DMN TDA DART
Builds on: TPT PVP CLIP CoOp MaPLE
Closest papers by meaning
- Improved Self-Training for Test-Time Adaptation (CVPR 2024 · similarity 0.87)
- Domain-Specific Block Selection and Paired-View Pseudo-Labeling for Online Test-Time Adaptation (CVPR 2024 · similarity 0.87)
- Efficient Test-time Adaptation through Candidate Verification and Divergence Shifts (NeurIPS 2026 · similarity 0.87)
- Efficient Test-Time Adaptation of Vision-Language Models (CVPR 2024 · similarity 0.87)
- Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimization (ICLR 2025 · similarity 0.87)
- Unified Entropy Optimization for Open-Set Test-Time Adaptation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.