Research map / Label-efficient and robust learning / Test-time adaptation

Free on the Fly: Enhancing Flexibility in Test-Time Adaptation with Online EM

CVPR 2025 · Label-efficient and robust learning › Test-time adaptation · approach: Test-time adaptation of vision-language models · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FreeTTA, a training-free test-time adaptation method for CLIP that runs an online EM over a Gaussian mixture of test features, initialized from text embeddings and weighted by zero-shot entropy.

Problem

Test-time adaptation of vision-language models under domain and distribution shifts, where existing methods either need costly optimization, store past test data, or treat test samples independently.

Approach

Models the test feature distribution as a Gaussian mixture with shared covariance, updated sample-by-sample via an online EM algorithm initialized with CLIP text embeddings, with each sample's update weighted by CLIP zero-shot prediction entropy. No gradient updates or stored test data are needed.

Key results

• With CLIP ViT-B/16, FreeTTA reaches 68.42% average accuracy on the 10 cross-domain datasets versus 64.59% zero-shot CLIP and 67.53% for TDA.
• On the ImageNet OOD benchmark with ViT-B/16, FreeTTA averages 64.42% versus 62.76% for ZERO and 63.89% for TDA.
• Ablation: removing VLM-prior weighting drops average accuracy from 68.42% to 67.78%, and fixing the mean vectors drops it to 64.64%.

Measured effects reported in the paper

Limitations

• Relies on the quality of zero-shot CLIP predictions as priors, so noisy initial predictions can drift the Gaussian parameters.
• Assumes Gaussian class-conditional distributions with a shared covariance matrix, which may not hold for all shifts.

Experimental setup

Datasets: FGVC Aircraft Caltech101 Stanford Cars DTD EuroSAT Flower102 Food101 Oxford-Pets SUN397 UCF101 ImageNet ImageNet-A

Benchmarks: FGVC Aircraft Caltech101 Stanford Cars DTD EuroSAT Flower102 Food101 Oxford-Pets SUN397 UCF101 ImageNet-A ImageNet-V2

Metrics: top-1 accuracy

Baselines: CLIP zero-shot CoOp CoCoOp TPT DiffTPT PromptAlign TDA MTA ZERO Tip-Adapter

Builds on: CLIP TPT TDA Gaussian discriminant analysis

Closest papers by meaning

• The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models (NeurIPS 2025 · similarity 0.90)
• Efficient Test-time Adaptation through Candidate Verification and Divergence Shifts (NeurIPS 2026 · similarity 0.90)
• Realistic Test-Time Adaptation of Vision-Language Models (CVPR 2025 · similarity 0.90)
• Noisy Test-Time Adaptation in Vision-Language Models (ICLR 2025 · similarity 0.89)
• Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimization (ICLR 2025 · similarity 0.89)
• DOTA: Distributional Test-time Adaptation of Vision-Language Models (NeurIPS 2025 · similarity 0.88)