Research map / Vision-language and multimodal / CLIP and zero-shot
Transferring Pre-trained Multimodal Representations with Cross-modal Similarity Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SwAV instead of SimCLR for student pre-training (ablation) | BeamCLIP RN50 · ImageNet-1K linear probe | top-1 accuracy: 74.8 -> 75.8 | Helped |
| replace cosine instance matching with cross-modal similarity matching (CE), SimCLR pre-trained (ablation) | BeamCLIP RN50 · CIFAR100 | accuracy: 48.26 -> 65.64 | Helped |
| add self-supervised pre-training of student (CE loss) (ablation) | BeamCLIP RN50 · Flowers102 | accuracy: 12.94 -> 62.18 | Helped |
| add self-supervised pre-training of student (CE loss) (ablation) | BeamCLIP RN50 · ImageNet-1K | accuracy: 63.30 -> 65.45 | Helped |
| add entropy minimization to CE loss (ablation) | BeamCLIP RN50 · ImageNet-1K | accuracy: 65.45 -> 66.23 | Helped |
| add entropy minimization to CE loss (ablation) | BeamCLIP RN50 · Flowers102 | accuracy: 62.18 -> 63.96 | Helped |
| add cosine instance similarity matching to CE+EntMin (ablation) | BeamCLIP RN50 · ImageNet-1K | accuracy: 66.23 -> 65.76 | Mixed |
| add context-based prompt augmentation (ablation) | BeamCLIP RN50 · Flowers102 | accuracy: 64.14 -> 75.86 | Helped |
| add context-based prompt augmentation (ablation) | BeamCLIP RN50 · CIFAR100 | accuracy: 66.18 -> 67.35 | Helped |
| BeamCLIP instead of KL-based KD | ResNet-50 student · ImageNet-1K | accuracy: 56.38 -> 66.17 | Helped |
| BeamCLIP instead of CRD contrastive distillation | ResNet-50 student · ImageNet-1K | accuracy: 30.90 -> 66.17 | Helped |
| increase number of text prompts from 20 to 100 (partial class names) | BeamCLIP CE+EntMin RN50 · CIFAR100 | accuracy: 21.72 -> 67.35 | Helped |
Closest papers by meaning
- Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP (ICLR 2024 · similarity 0.88)
- SmartCLIP: Modular Vision-language Alignment with Identification Guarantees (CVPR 2025 · similarity 0.88)
- SoftCLIP: Softer Cross-Modal Alignment Makes CLIP Stronger (AAAI 2024 · similarity 0.87)
- Preserving Clusters in Prompt Learning for Unsupervised Domain Adaptation (CVPR 2025 · similarity 0.87)
- MmAP: Multi-Modal Alignment Prompt for Cross-Domain Multi-Task Learning (AAAI 2024 · similarity 0.87)
- Adaptive Prompt-Based Semantic Embedding with Inspire Potential of Implicit Knowledge for Cross-Modal Retrieval (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.