AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

Learning Visual N-Grams From Web Data

Ang Li, Allan Jabri, Armand Joulin, Laurens van der Maaten

ICCV 2017 · Vision-language and multimodal › CLIP and zero-shot · approach: Open-vocabulary alignment from VLMs · 94 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use Jelinek-Mercer loss instead of naive n-gram loss, stupid back-off at test (ablation)visual n-gram model · YFCC100M testperplexity: 297 -> 276Improved
use Jelinek-Mercer loss instead of naive n-gram loss, J-M smoothing at test (ablation)visual n-gram model · YFCC100M testperplexity: 212 -> 199Improved
add Jelinek-Mercer smoothing at prediction time instead of stupid back-off (ablation)Jelinek-Mercer loss model · YFCC100M testperplexity: 276 -> 199Improved
use Jelinek-Mercer loss instead of naive n-gram loss (ablation)visual n-gram model · YFCC100M testphrase recall@1: 5.5 -> 6.2Improved
use Jelinek-Mercer loss instead of naive n-gram loss (ablation)visual n-gram model · YFCC100M testphrase accuracy (human raters): 36.4 -> 42.0Improved
train the full network with n-gram loss instead of linear layer on Imagenet featuresImagenet + linear baseline · YFCC100M testphrase recall@1: 5.0 -> 6.2Improved
include longer n-grams (up to five) in the dictionary (ablation)Jelinek-Mercer model · YFCC100M testrecall@k of n-gram retrievalImproved
use Jelinek-Mercer loss instead of naive n-gram loss (ablation)visual n-gram model · YFCC100M testcaption retrieval accuracy (human raters): 42.0 -> 53.1Improved
use n-gram dictionary models instead of unigram model (ablation)Jelinek-Mercer model · YFCC100M testcaption retrieval recall@1: 1.2 -> 7.1Improved
finetune Jelinek-Mercer model on COCOJelinek-Mercer model · COCO-5K image retrievalrecall@1: 5.0 -> 11.0Improved
finetune Jelinek-Mercer model on COCOJelinek-Mercer model · COCO-5K caption retrievalrecall@1: 8.7 -> 17.8Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy