Research map / Vision-language and multimodal / CLIP and zero-shot
SILC: Improving Vision Language Pretraining with Self-Distillation
Muhammad Ferjad Naeem, Yongqin Xian, Xiaohua Zhai, Lukas Hoyer, Luc Van Gool, Federico Tombari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add local-to-global self-distillation to CLIP | CLIP (WebLI) ViT-B/16 · ImageNet zero-shot | top-1 accuracy: 74.1 -> 75.3 | Improved |
| add self-distillation to SigLIP | SigLIP ViT-B/16 · ImageNet zero-shot | top-1 accuracy: 75.1 -> 75.8 | Improved |
| add self-distillation to SigLIP | SigLIP ViT-B/16 · CIFAR100 zero-shot | top-1 accuracy: 69.8 -> 69.2 | Mixed |
| add self-distillation to CLIP | CLIP (WebLI) · A-150 zero-shot segmentation | mIoU: 15.0 -> 17.2 | Improved |
| finetune SILC* on cleaner WebLI subset (ablation) | SILC-C* · COCO retrieval | I2T@1: 62.5 -> 66.1 | Improved |
| use SILC backbone in CAT-Seg instead of CLIP | CAT-Seg ViT-B/16 · A-150 open-vocab segmentation | mIoU: 27.2 -> 36.6 | Improved |
| initialize OWLv2 from SILC-S* rather than SigLIP | OWLv2 · COCO zero-shot detection | AP: 40.9 -> 42.7 | Improved |
| use SILC visual features in LiT-Decoder | SigLIP baseline · COCO captioning | CIDEr: 117.5 -> 121.2 | Improved |
| add additional image views to CLIP (ablation) | CLIP (WebLI) · Zero-shot segmentation A-150 | mIoU: 11.8 -> 11.7 | No clear effect |
| add additional views to CLIP (ablation) | CLIP (WebLI) · ImageNet zero-shot | top-1 accuracy: 71.7 -> 73.6 | Improved |
| add EMA over model weights (ablation) | CLIP + additional views · ImageNet zero-shot | top-1 accuracy: 73.6 -> 73.7 | No clear effect |
| add self-distillation from local crops on top of views and EMA (ablation) | CLIP + views + EMA · Open vocab seg PC-59 | mIoU: 57.5 -> 60.7 | Improved |
Closest papers by meaning
- Building Vision-Language Models on Solid Foundations with Masked Distillation (CVPR 2024 · similarity 0.92)
- Post-pre-training for Modality Alignment in Vision-Language Foundation Models (CVPR 2025 · similarity 0.90)
- Modeling Caption Diversity in Contrastive Vision-Language Pretraining (ICML 2024 · similarity 0.90)
- FFF: Fixing Flawed Foundations in Contrastive Pre-Training Results in Very Strong Vision-Language Models (CVPR 2024 · similarity 0.89)
- Text and Image Are Mutually Beneficial: Enhancing Training-Free Few-Shot Classification with CLIP (AAAI 2025 · similarity 0.89)
- Interpreting and Analysing CLIP's Zero-Shot Image Classification via Mutual Knowledge (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
