Research map / Vision-language and multimodal / CLIP and zero-shot
LiT: Zero-Shot Transfer With Locked-Image Text Tuning
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Lock the pretrained image tower and train a text tower to align with it (ablation) | Contrastive tuning · ImageNet | zero-shot accuracy: 57.2 -> 70.1 | Helped |
| Lock the pretrained image tower and train a text tower to align with it (ablation) | Contrastive tuning · CIFAR100 | zero-shot accuracy: 62.1 -> 70.9 | Helped |
| Initialize the image tower from pretrained weights rather than training both towers from scratch (ablation) | Contrastive tuning · ImageNet | zero-shot accuracy: 50.6 -> 70.1 | Helped |
| Use global rather than per-device contrastive loss (ablation) | Contrastive tuning · ImageNet zero-shot transfer and retrieval | performance | Helped |
| Increase text tower capacity (ablation) | LiT · ImageNet zero-shot transfer | accuracy: 61.1% -> 67.6% | Helped |
| Increase image tower capacity (ablation) | LiT · ImageNet zero-shot transfer | accuracy: 74.5% -> 81.2% | Helped |
| Use all YFCC100m text signals jointly rather than sampling one signal (ablation) | LiT on YFCC100m · ImageNet classification and image-text retrieval | performance | Helped |
| Increase maximum text length from 16 to 32 tokens (ablation) | LiT on YFCC100m · ImageNet classification and retrieval | performance | No clear effect |
| Deduplicate upstream data against downstream train and test examples (ablation) | LiT · ImageNet and MSCOCO retrieval | performance: 70.2; 70.2; 69.9 | No clear effect |
| Initialize the text tower from pretrained language-model weights (ablation) | LiT on YFCC100m-CLIP · ImageNet zero-shot accuracy and MSCOCO retrieval | performance: +1.4; +1.6; +1.2 | Helped |
| Use BERT rather than the ViT text encoder (ablation) | LiT on YFCC100m-CLIP · ImageNet and MSCOCO retrieval | performance: 58.8; 35.2; 20.0 | Mixed |
| Train on full multilingual YFCC100m instead of the English CLIP subset (ablation) | LiT multilingual models · Non-English zero-shot classification and WIT retrieval | performance | Helped |
Closest papers by meaning
- Lipsum-FT: Robust Fine-Tuning of Zero-Shot Models Using Random Text Guidance (ICLR 2024 · similarity 0.86)
- Text and Image Are Mutually Beneficial: Enhancing Training-Free Few-Shot Classification with CLIP (AAAI 2025 · similarity 0.86)
- Text-Guided Attention is All You Need for Zero-Shot Robustness in Vision-Language Models (NeurIPS 2024 · similarity 0.85)
- Zero-Shot Adaptation of Parameter-Efficient Fine-Tuning in Diffusion Models (ICML 2025 · similarity 0.85)
- Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP (ICLR 2024 · similarity 0.85)
- Context-Aware Multimodal Pretraining (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.