Research map / Vision-language and multimodal / CLIP and zero-shot
GRIT-VLP: Grouped Mini-Batch Sampling for Efficient Vision and Language Pre-training
Jaeseok Byun, Taebaek Hwang, Jianlong Fu, Taesup Moon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use hard instead of random in-batch negatives for ITM (no ITC) (ablation) | ALBEF_Base (10 epochs) · MS-COCO TR R@1 | R@1: 61.6 -> 68.6 | Improved |
| add ITC to ITM_rand (ablation) | ALBEF_Base (10 epochs) · MS-COCO TR R@1 | R@1: 61.6 -> 66.8 | Improved |
| add ITC to ITM_hard (ablation) | ALBEF_Base (10 epochs) · MS-COCO TR R@1 | R@1: 68.6 -> 72.3 | Improved |
| raise MLM masking probability from 15% to 50% (ablation) | ALBEF_Base (20 epochs) · MS-COCO TR R@1 | R@1: 73.8 -> 75.6 | Improved |
| raise MLM masking probability from 15% to 50% (ablation) | ALBEF_Base (20 epochs) · NLVR2 test | accuracy: 80.37 -> 80.54 | No clear effect |
| add GRIT with shuffling to ALBEF_Base50 (ablation) | ALBEF_Base50 · MS-COCO TR R@1 | R@1: 75.6 -> 76.4 | Improved |
| add GRIT with shuffling to ALBEF_Base50 (ablation) | ALBEF_Base50 · NLVR2 test | accuracy: 80.54 -> 81.32 | Improved |
| remove both shufflings from GRIT (ablation) | GRIT · MS-COCO TR R@1 | R@1: 76.4 -> 74.7 | Hurt |
| use naive GRIT instead of efficient GRIT (ablation) | GRIT · epoch time | time per epoch: 2h 30m -> 3h | Hurt |
| add ITC consistency loss to GRIT at mask 50 (ablation) | GRIT-VLP · MS-COCO TR R@1 | R@1: 76.4 -> 77.1 | Improved |
| add GRIT to ITC-only small model (ablation) | ITC dual encoder · MS-COCO TR R@1 | R@1: 59.7 -> 63.0 | Improved |
| use queue-based ITC instead of plain ITC (ablation) | ITC dual encoder · MS-COCO TR R@1 | R@1: 59.7 -> 60.3 | No clear effect |
Later papers that compared with this method
2 later accepted papers report GRIT-VLP as a baseline in their results tables; 2 of them report a better number on most of the results they share with it.
- Concept-Aware Batch Sampling Improves Language-Image Pretraining (CVPR 2026 · better on 4 of 4 shared results)
- FALCON: False-Negative Aware Learning of Contrastive Negatives in Vision-Language Alignment (CVPR 2026 · better on 3 of 3 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- MAFA: Managing False Negatives for Vision-Language Pre-training (CVPR 2024 · similarity 0.89)
- Enhancing Fine-Grained Vision-Language Pretraining with Negative Augmented Samples (AAAI 2025 · similarity 0.89)
- Language Models as Black-Box Optimizers for Vision-Language Models (CVPR 2024 · similarity 0.88)
- Concept-Aware Batch Sampling Improves Language-Image Pretraining (CVPR 2026 · similarity 0.88)
- Probing Visual Language Priors in VLMs (ICML 2025 · similarity 0.87)
- Connecting the Dots: Collaborative Fine-tuning for Black-Box Vision-Language Models (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
