Research map / Vision-language and multimodal / CLIP and zero-shot
HQ-CLIP: Leveraging Large Vision-Language Models to Create High-Quality Image-Text Datasets and CLIP Models
Zhixiang Wei, Guangting Wang, Xiaoxiao Ma, Ke Mei, Huaian Chen, Yi Jin, Fengyun Rao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on VLM-150M recaptioned data (mixed training) (ablation) | DFN CLIP baseline · DataComp Medium | avg over 38 datasets: 36.8 -> 39.9 | Improved |
| add hard negative identification loss (ablation) | mixed-training CLIP · DataComp Medium | avg over 38 datasets: 39.9 -> 40.7 | Improved |
| add short-tag classification loss (ablation) | CLIP with HNI · DataComp Medium | avg over 38 datasets: 40.7 -> 41.1 | Improved |
| train HQ-CLIP instead of plain training on VLM-150M (ablation) | VLM-150M CLIP · DataComp Large | avg over 38 datasets: 56.6 -> 58.6 | Improved |
| train on VLM-150M with plain CLIP loss | DFN CLIP baseline · DataComp Large | ImageNet: 68.7 -> 67.7 | Hurt |
| use 100% enriched text instead of 75% mix with raw captions (ablation) | mixed training · DataComp Medium | avg over 38 datasets: 39.9 -> 36.9 | Hurt |
| mix raw captions with enriched text (75% enriched vs 0%) (ablation) | DFN CLIP baseline · DataComp Medium | avg over 38 datasets: 36.8 -> 39.9 | Improved |
| increase hard negatives per image from 1 to 4 (ablation) | HQ-CLIP · DataComp Medium | avg over 38 datasets: 40.1 -> 39.9 | No clear effect |
| vary STC tag vocabulary size from 3000 to 90000 (ablation) | HQ-CLIP · DataComp Medium | avg over 38 datasets: 39.9 -> 40.5 | No clear effect |
| add caption input to Qwen2-VL-7B SFT refinement (ablation) | Qwen2-VL-7B captioner · DataComp Medium | avg over 38 datasets: 38.7 -> 39.9 | Improved |
| scale captioner from 7B (SFT) to 72B (ablation) | Qwen2-VL captioner · DataComp Medium | avg over 38 datasets: 39.9 -> 40.1 | No clear effect |
| use HQ-CLIP vision encoder instead of DFN in LLaVA-1.5 | LLaVA-1.5 ViT-B · LLaVA-1.5 | MME: 1452.1 -> 1574.0 | Improved |
Later papers that compared with this method
1 later accepted paper reports HQ-CLIP as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Dynamic Cluster Data Sampling for Efficient and Long-Tail-Aware Vision-Language Pre-training (ECCV 2026 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Diffusion Feedback Helps CLIP See Better (ICLR 2025 · similarity 0.92)
- Building Vision-Language Models on Solid Foundations with Masked Distillation (CVPR 2024 · similarity 0.91)
- Demystifying CLIP Data (ICLR 2024 · similarity 0.91)
- Post-pre-training for Modality Alignment in Vision-Language Foundation Models (CVPR 2025 · similarity 0.91)
- un$^2$CLIP: Improving CLIP's Visual Detail Capturing Ability via Inverting unCLIP (NeurIPS 2025 · similarity 0.91)
- Meta CLIP 2: A Worldwide Scaling Recipe (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
