AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

Noise-Aware Learning from Web-Crawled Image-Text Data for Image Captioning

Wooyoung Kang, Jonghwan Mun, Sungjun Lee, Byungseok Roh

ICCV 2023 · Vision-language and multimodal › CLIP and zero-shot · approach: Recaptioning and text-supervised pretraining · 24 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train with alignment-level control signal instead of plain training on all noisy datacaptioner on CC3M · MSCOCO zero-shotCIDEr: 47.56 -> 62.04Improved
use NoC instead of CLIP-similarity filteringcaptioner on CC3M · MSCOCO zero-shotCIDEr: 54.66 -> 62.04Improved
filter training pairs by CLIP similarityVanilla captioner · MSCOCO zero-shotCIDEr: 47.56 -> 54.66Improved
re-weight loss by CLIP similarityVanilla captioner · MSCOCO zero-shotCIDEr: 47.56 -> 50.86Improved
use highest alignment level z=7 versus z=8 at inference (ablation)NoC · MSCOCO zero-shotCIDEr: 62.04 -> 53.50Hurt
use low alignment level z at inference (z=1 vs z=7) (ablation)NoC · MSCOCO zero-shotCIDEr: 62.04 -> 52.78Hurt
apply NoC to the pre-train then fine-tune pipelineCC3M-pretrained captioner · MSCOCO test (fine-tuned)CIDEr: 126.14 -> 129.09Improved
use uniform instead of quantile bucketing (ablation)NoC · MSCOCO zero-shotCLIPScore: 65.72 -> 66.65Improved
use uniform instead of quantile bucketing (ablation)NoC · MSCOCO zero-shotCIDEr: 49.22 -> 49.18No clear effect
increase number of bucket bins from 8 to 16 (ablation)NoC · MSCOCO zero-shotCIDEr: 49.18 -> 48.58Hurt
NoC generated captions as retrieval queries instead of ground-truth captionsself-retrieval · MSCOCO self-retrievalR@1: 34.57 -> 40.00Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy