AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

SILC: Improving Vision Language Pretraining with Self-Distillation

Muhammad Ferjad Naeem, Yongqin Xian, Xiaohua Zhai, Lukas Hoyer, Luc Van Gool, Federico Tombari

ECCV 2024 · Vision-language and multimodal › CLIP and zero-shot · approach: CLIP training recipes and data curation · 45 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add local-to-global self-distillation to CLIPCLIP (WebLI) ViT-B/16 · ImageNet zero-shottop-1 accuracy: 74.1 -> 75.3Improved
add self-distillation to SigLIPSigLIP ViT-B/16 · ImageNet zero-shottop-1 accuracy: 75.1 -> 75.8Improved
add self-distillation to SigLIPSigLIP ViT-B/16 · CIFAR100 zero-shottop-1 accuracy: 69.8 -> 69.2Mixed
add self-distillation to CLIPCLIP (WebLI) · A-150 zero-shot segmentationmIoU: 15.0 -> 17.2Improved
finetune SILC* on cleaner WebLI subset (ablation)SILC-C* · COCO retrievalI2T@1: 62.5 -> 66.1Improved
use SILC backbone in CAT-Seg instead of CLIPCAT-Seg ViT-B/16 · A-150 open-vocab segmentationmIoU: 27.2 -> 36.6Improved
initialize OWLv2 from SILC-S* rather than SigLIPOWLv2 · COCO zero-shot detectionAP: 40.9 -> 42.7Improved
use SILC visual features in LiT-DecoderSigLIP baseline · COCO captioningCIDEr: 117.5 -> 121.2Improved
add additional image views to CLIP (ablation)CLIP (WebLI) · Zero-shot segmentation A-150mIoU: 11.8 -> 11.7No clear effect
add additional views to CLIP (ablation)CLIP (WebLI) · ImageNet zero-shottop-1 accuracy: 71.7 -> 73.6Improved
add EMA over model weights (ablation)CLIP + additional views · ImageNet zero-shottop-1 accuracy: 73.6 -> 73.7No clear effect
add self-distillation from local crops on top of views and EMA (ablation)CLIP + views + EMA · Open vocab seg PC-59mIoU: 57.5 -> 60.7Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy