Research map / Vision-language and multimodal / CLIP and zero-shot
Learning Transferable Visual Models From Natural Language Supervision
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on image-text pairs with a contrastive matching objective instead of the Visual N-Grams approach | CLIP · ImageNet zero-shot | accuracy: 11.5 -> 76.2 | Helped |
| Use the prompt “A photo of a {label}.” instead of contextless class names (ablation) | CLIP zero-shot classifier · ImageNet | accuracy: +1.3% | Helped |
| Ensemble 80 context prompts instead of using the single default prompt (ablation) | CLIP zero-shot classifier · ImageNet | accuracy: +3.5% | Helped |
| Combine prompt engineering and prompt ensembling instead of embedding class names without context (ablation) | CLIP zero-shot classifier · 36 datasets | average accuracy: almost 5 points | Helped |
| Replace caption language modeling with bag-of-words text prediction (ablation) | CLIP image-caption baseline · ImageNet zero-shot transfer efficiency | transfer learning efficiency: 3x | Helped |
| Replace predictive bag-of-words objective with contrastive image-text matching (ablation) | CLIP · ImageNet zero-shot transfer efficiency | transfer learning efficiency: 4x | Helped |
| Adapt the zero-shot CLIP classifier to ImageNet using supervised training (ablation) | CLIP · ImageNet | accuracy: 9.2% to 85.4% | Helped |
| Adapt the zero-shot CLIP classifier to ImageNet using supervised training (ablation) | CLIP · Seven natural distribution-shift datasets | average accuracy | Hurt |
| Use dataset-specific zero-shot class names instead of an ImageNet classifier with pooled class mappings (ablation) | CLIP · Seven natural distribution-shift datasets | effective robustness: 5% | Helped |
| Increase the number of ImageNet examples used to fit a classifier on CLIP features (ablation) | CLIP few-shot classifier · Natural distribution shifts | effective robustness | Hurt |
| Use ViT image encoders instead of CLIP ResNet image encoders (ablation) | CLIP · Linear-probe evaluation | compute efficiency: 3x | Helped |
| Increase CLIP model compute (ablation) | CLIP · 39 evaluations across 36 datasets | zero-shot error: 44x range of compute | Helped |
Closest papers by meaning
- Linearly Mapping from Image to Text Space (ICLR 2023 · similarity 0.91)
- InstructCV: Instruction-Tuned Text-to-Image Diffusion Models as Vision Generalists (ICLR 2024 · similarity 0.88)
- Distilling Vision-Language Models on Millions of Videos (CVPR 2024 · similarity 0.88)
- Explanatory Instructions: Towards Unified Vision Tasks Understanding and Zero-shot Generalization (ICML 2025 · similarity 0.88)
- Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models (NeurIPS 2025 · similarity 0.88)
- A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.