Research map / Vision-language and multimodal / CLIP and zero-shot
S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions
Sangwoo Mo, Minkyu Kim, Kyungmin Lee, Jinwoo Shin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add caption-level pseudo-labels (ablation) | CLIP (fine-tune) · avg zero-shot accuracy | accuracy: 64.7 -> 69.6 | Improved |
| add keyword-level pseudo-labels (ablation) | CLIP (fine-tune) · avg zero-shot accuracy | accuracy: 64.7 -> 70.0 | Improved |
| combine caption-level and keyword-level pseudo-labels (ablation) | S-CLIP · avg zero-shot accuracy | accuracy: 70.0 -> 71.9 | Improved |
| caption-level vs keyword-level objectives on retrieval (ablation) | S-CLIP · avg retrieval R@1 | R@1: 10.5 -> 9.9 | Mixed |
| increase Sinkhorn iterations from 0 to 10 (ablation) | S-CLIP · avg zero-shot accuracy | accuracy: 70.6 -> 71.9 | Improved |
| increase Sinkhorn iterations from 0 to 10 (ablation) | S-CLIP · avg retrieval R@1 | R@1: 9.2 -> 10.6 | Improved |
| use YAKE-200 keywords instead of class names (ablation) | S-CLIP · avg zero-shot accuracy | accuracy: 71.9 -> 72.2 | Mixed |
| use pseudo-embedding instead of pseudo-label (ablation) | S-CLIP · avg zero-shot accuracy | accuracy: 71.9 -> 68.8 | Hurt |
| compute OT with text embeddings instead of image embeddings (ablation) | S-CLIP · avg zero-shot accuracy | accuracy: 71.9 -> 71.6 | No clear effect |
| use hard-max instead of soft-max partial label loss (ablation) | S-CLIP · avg zero-shot accuracy | accuracy: 71.9 -> 66.1 | Hurt |
| Hard-PL naive pseudo-labeling vs supervised fine-tune | CLIP · RSICD-CLS zero-shot, L=U | accuracy: 58.3 -> 56.6 | Hurt |
| Soft-PL under distribution shift (L!=U) vs supervised fine-tune | CLIP · RSICD-CLS zero-shot | accuracy: 58.3 -> 56.1 | Hurt |
Later papers that compared with this method
6 later accepted papers report SCLIP as a baseline in their results tables; 5 of them report a better number on most of the results they share with it.
- Geometry-Preserving Unsupervised Alignment for Heterogeneous Foundation Models (ICML 2026 · better on 2 of 2 shared results)
- SOTAlign: Semi-Supervised Alignment of Unimodal Vision and Language Models via Optimal Transport (ICML 2026 · better on 1 of 1 shared result)
- Brain Mapping with Dense Features: Grounding Cortical Semantic Selectivity in Natural Images With Vision Transformers (ICLR 2025 · better on 6 of 8 shared results)
- Assessing and Learning Alignment of Unimodal Vision and Language Models (CVPR 2025 · better on 3 of 3 shared results)
- ResCLIP: Residual Attention for Training-free Dense Vision-language Inference (CVPR 2025 · better on 3 of 3 shared results)
- COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training (CVPR 2025 · better on 0 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- DiffCLIP: Few-shot Language-driven Multimodal Classifier (AAAI 2025 · similarity 0.89)
- Demystifying CLIP Data (ICLR 2024 · similarity 0.89)
- SuperCLIP: CLIP with Simple Classification Supervision (NeurIPS 2025 · similarity 0.89)
- Building Vision-Language Models on Solid Foundations with Masked Distillation (CVPR 2024 · similarity 0.88)
- Modeling Caption Diversity in Contrastive Vision-Language Pretraining (ICML 2024 · similarity 0.88)
- Quality-Driven Curation of Remote Sensing Vision-Language Data via Learned Scoring Models (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
