Research map / Vision-language and multimodal / CLIP and zero-shot
DataComp: In search of the next generation of multimodal datasets
Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, Eyal Orgad, Rahim Entezari and 22 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| filter the pool with image-based intersect CLIP score instead of no filtering | CLIP ViT-L/14 on CommonPool · ImageNet, xlarge | accuracy: 0.723 -> 0.792 | Improved |
| use CLIP score (L/14 30%) filtering instead of no filtering | CLIP on CommonPool · ImageNet, xlarge | accuracy: 0.723 -> 0.764 | Improved |
| add image-based filtering on top of CLIP score filtering | CLIP on CommonPool · ImageNet, large | accuracy: 0.578 -> 0.631 | Improved |
| add image-based filtering on top of CLIP score filtering | CLIP on CommonPool · Retrieval, small | accuracy: 0.119 -> 0.094 | Hurt |
| train on DataComp-1B instead of OpenAI WIT, same compute | CLIP ViT-L/14 · ImageNet zero-shot | accuracy (%): 75.5 -> 79.2 | Improved |
| train on DataComp-1B instead of LAION-2B | CLIP ViT-L/14 · ImageNet zero-shot | accuracy (%): 73.1 -> 79.2 | Improved |
| add external BYOD data (CC12M, YFCC15M, RedCaps, Shutterstock) to CLIP-filtered pool | CLIP at large scale · ImageNet | accuracy: +4.3 percentage points | Improved |
| use random subsets of pool instead of full pool | CLIP on CommonPool · ImageNet | accuracy | Hurt |
| keep top ~30% by CLIP score rather than other fractions | CLIP score filtering · ImageNet | accuracy | Improved |
| apply LAION-style filtering on CommonPool versus LAION-2B itself | CLIP large scale · ImageNet | accuracy (%): 55.3 vs 55.7 | No clear effect |
| add face blurring to training images (ablation) | CLIP medium scale, CLIP score (B/32, 30%) · ImageNet | accuracy: 0.287 -> 0.282 | No clear effect |
| add face blurring to training images (ablation) | CLIP medium scale, CLIP score (B/32, thresh 0.3)+English · ImageNet | accuracy: 0.209 -> 0.196 | Hurt |
Closest papers by meaning
- CLIPLoss and Norm-Based Data Selection Methods for Multimodal Contrastive Learning (NeurIPS 2024 · similarity 0.87)
- Effective pruning of web-scale datasets based on complexity of concept clusters (ICLR 2024 · similarity 0.87)
- Data-Juicer Sandbox: A Feedback-Driven Suite for Multimodal Data-Model Co-development (ICML 2025 · similarity 0.87)
- Demystifying CLIP Data (ICLR 2024 · similarity 0.87)
- Diffusion Feedback Helps CLIP See Better (ICLR 2025 · similarity 0.87)
- CtrlSynth: Controllable Image Text Synthesis for Data-Efficient Multimodal Learning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
