AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions

Sangwoo Mo, Minkyu Kim, Kyungmin Lee, Jinwoo Shin

NeurIPS 2023 · Vision-language and multimodal › CLIP and zero-shot · approach: CLIP training recipes and data curation · 5 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add caption-level pseudo-labels (ablation)CLIP (fine-tune) · avg zero-shot accuracyaccuracy: 64.7 -> 69.6Improved
add keyword-level pseudo-labels (ablation)CLIP (fine-tune) · avg zero-shot accuracyaccuracy: 64.7 -> 70.0Improved
combine caption-level and keyword-level pseudo-labels (ablation)S-CLIP · avg zero-shot accuracyaccuracy: 70.0 -> 71.9Improved
caption-level vs keyword-level objectives on retrieval (ablation)S-CLIP · avg retrieval R@1R@1: 10.5 -> 9.9Mixed
increase Sinkhorn iterations from 0 to 10 (ablation)S-CLIP · avg zero-shot accuracyaccuracy: 70.6 -> 71.9Improved
increase Sinkhorn iterations from 0 to 10 (ablation)S-CLIP · avg retrieval R@1R@1: 9.2 -> 10.6Improved
use YAKE-200 keywords instead of class names (ablation)S-CLIP · avg zero-shot accuracyaccuracy: 71.9 -> 72.2Mixed
use pseudo-embedding instead of pseudo-label (ablation)S-CLIP · avg zero-shot accuracyaccuracy: 71.9 -> 68.8Hurt
compute OT with text embeddings instead of image embeddings (ablation)S-CLIP · avg zero-shot accuracyaccuracy: 71.9 -> 71.6No clear effect
use hard-max instead of soft-max partial label loss (ablation)S-CLIP · avg zero-shot accuracyaccuracy: 71.9 -> 66.1Hurt
Hard-PL naive pseudo-labeling vs supervised fine-tuneCLIP · RSICD-CLS zero-shot, L=Uaccuracy: 58.3 -> 56.6Hurt
Soft-PL under distribution shift (L!=U) vs supervised fine-tuneCLIP · RSICD-CLS zero-shotaccuracy: 58.3 -> 56.1Hurt

Later papers that compared with this method

6 later accepted papers report SCLIP as a baseline in their results tables; 5 of them report a better number on most of the results they share with it.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy