Research map / Vision-language and multimodal / CLIP and zero-shot
Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| mix YFCC and LAION 50:50 (15M total) instead of training on the better single source | CLIP ResNet-50 · ImageNet distribution shifts (ImageNet-R, Sketch) | effective robustness (linear trend) | Hurt |
| increase mixture size to 30M using all YFCC and LAION data (ablation) | CLIP ResNet-50 input mixing · ImageNet distribution shifts | effective robustness (linear trend) | No clear effect |
| vary YFCC share of a fixed 15M mixture from 15M to 0M (ablation) | CLIP ResNet-50 input mixing · ImageNet distribution shifts | effective robustness (linear trend) | Mixed |
| ensemble logits of separately trained YFCC and LAION models instead of training on mixed data (ablation) | CLIP ResNet-50 · ImageNet distribution shifts | effective robustness (linear trend) | No clear effect |
| choose among six data sources (YFCC, LAION, CC12M, RedCaps, Shutterstock, WIT) as pre-training data | CLIP ResNet-50 · ImageNet-V2, R, Sketch, ObjectNet | effective robustness | Mixed |
| filter a noisy source with an existing robust pre-trained model (theory) | Gaussian binary classification model · theoretical model | slope of robustness trend | Helped |
Closest papers by meaning
- What Makes CLIP More Robust to Long-Tailed Pre-Training Data? A Controlled Study for Transferable Insights (NeurIPS 2024 · similarity 0.90)
- Does CLIP’s generalization performance mainly stem from high train-test similarity? (ICLR 2024 · similarity 0.89)
- CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning (NeurIPS 2024 · similarity 0.89)
- Meta CLIP 2: A Worldwide Scaling Recipe (NeurIPS 2025 · similarity 0.88)
- Demystifying CLIP Data (ICLR 2024 · similarity 0.88)
- CLIP the Bias: How Useful is Balancing Data in Multimodal Learning? (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.