Research map / Vision-language and multimodal / Human and embodied modeling
ECCV Caption: Correcting False Negatives by Collecting Machine-and-Human-Verified Image-Caption Associations for MS-COCO
Sanghyuk Chun, Wonjae Kim, Song Park, Minsuk Chang, Seong Joon Oh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| evaluate with ECCV mAP@R instead of COCO 1K R@1 | CLIP ViT-L/14 vs PCME · ECCV Caption | ranking: 55.4% vs 40.1% COCO 1K R@1 -> 28.0% vs 37.1% ECCV mAP@R | Mixed |
| use semi-hard instead of hardest negative mining (ablation) | PVSE K=1 · ECCV Caption | mAP@R: 35.76 -> 36.63 | Improved |
| add hardest negative mining (ablation) | PVSE K=1 · COCO 1K | R@1: 43.28 -> 49.12 | Improved |
| add hardest negative mining (vs none) (ablation) | PVSE K=1 · ECCV Caption | mAP@R: 33.34 -> 35.76 | Improved |
| add semi-hard negative mining (vs none) (ablation) | PVSE K=1 · ECCV Caption | mAP@R: 33.34 -> 36.63 | Improved |
| increase PVSE embeddings K from 1 to 2 (ablation) | PVSE · ECCV Caption | mAP@R: 33.98 -> 40.26 | Improved |
| add CutMix pre-training (ablation) | PCME · ECCV Caption | mAP@R: 37.11 -> 41.74 | Improved |
| fine-tune VinVL on COCO Caption (ablation) | VinVL · ECCV Caption | mAP@R: 22.18 -> 40.81 | Improved |
| fine-tune ViLT on COCO Caption (ablation) | ViLT · ECCV Caption | mAP@R: 26.84 -> 34.58 | Improved |
| use COCO 1K R@1 ranking to predict ECCV mAP@R ranking | 25 VL models · rank correlation | Kendall tau: 0.47 | Hurt |
| use PMRP ranking to predict ECCV mAP@R ranking | 25 VL models · rank correlation | Kendall tau: 0.20 | Hurt |
| replace R@1 with mAP@R as metric in human study | metric · human preference alignment | preference: A: 70.85, B: 10.66, C: 13.15, D: 4.89, E: 0.44 | Improved |
Closest papers by meaning
- Improved Probabilistic Image-Text Representations (ICLR 2024 · similarity 0.87)
- LEMoN: Label Error Detection using Multimodal Neighbors (ICML 2025 · similarity 0.86)
- MASS: Overcoming Language Bias in Image-Text Matching (AAAI 2025 · similarity 0.86)
- A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions (CVPR 2024 · similarity 0.86)
- CAPability: A Comprehensive Visual Caption Benchmark for Evaluating Both Correctness and Thoroughness (NeurIPS 2025 · similarity 0.86)
- Visual Fact Checker: Enabling High-Fidelity Detailed Caption Generation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
