Research map / Vision-language and multimodal / CLIP and zero-shot
Noise-Aware Learning from Web-Crawled Image-Text Data for Image Captioning
Wooyoung Kang, Jonghwan Mun, Sungjun Lee, Byungseok Roh
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with alignment-level control signal instead of plain training on all noisy data | captioner on CC3M · MSCOCO zero-shot | CIDEr: 47.56 -> 62.04 | Improved |
| use NoC instead of CLIP-similarity filtering | captioner on CC3M · MSCOCO zero-shot | CIDEr: 54.66 -> 62.04 | Improved |
| filter training pairs by CLIP similarity | Vanilla captioner · MSCOCO zero-shot | CIDEr: 47.56 -> 54.66 | Improved |
| re-weight loss by CLIP similarity | Vanilla captioner · MSCOCO zero-shot | CIDEr: 47.56 -> 50.86 | Improved |
| use highest alignment level z=7 versus z=8 at inference (ablation) | NoC · MSCOCO zero-shot | CIDEr: 62.04 -> 53.50 | Hurt |
| use low alignment level z at inference (z=1 vs z=7) (ablation) | NoC · MSCOCO zero-shot | CIDEr: 62.04 -> 52.78 | Hurt |
| apply NoC to the pre-train then fine-tune pipeline | CC3M-pretrained captioner · MSCOCO test (fine-tuned) | CIDEr: 126.14 -> 129.09 | Improved |
| use uniform instead of quantile bucketing (ablation) | NoC · MSCOCO zero-shot | CLIPScore: 65.72 -> 66.65 | Improved |
| use uniform instead of quantile bucketing (ablation) | NoC · MSCOCO zero-shot | CIDEr: 49.22 -> 49.18 | No clear effect |
| increase number of bucket bins from 8 to 16 (ablation) | NoC · MSCOCO zero-shot | CIDEr: 49.18 -> 48.58 | Hurt |
| NoC generated captions as retrieval queries instead of ground-truth captions | self-retrieval · MSCOCO self-retrieval | R@1: 34.57 -> 40.00 | Improved |
Closest papers by meaning
- Noise-Aware Image Captioning with Progressively Exploring Mismatched Words (AAAI 2024 · similarity 0.90)
- Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Training (AAAI 2024 · similarity 0.88)
- Painting with Words: Elevating Detailed Image Captioning with Benchmark and Alignment Learning (ICLR 2025 · similarity 0.86)
- CapsFusion: Rethinking Image-Text Data at Scale (CVPR 2024 · similarity 0.86)
- Patch Matters: Training-free Fine-grained Image Caption Enhancement via Local Perception (CVPR 2025 · similarity 0.86)
- Improving Cross-Modal Alignment with Synthetic Pairs for Text-Only Image Captioning (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
