AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Learning to Embed Semantic Similarity for Joint Image-Text Retrieval

Noam Malali, Yosi Keller

TPAMI 2021 · Vision-language and multimodal › Human and embodied modeling · approach: Composed and text-based image retrieval · 10 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace text-only center loss with semantic centers on both modalities (ablation)joint embedding · Flickr8Kannotation r@1: 6.6 -> 30.1Improved
replace text-only center loss with semantic centers (ablation)joint embedding · Flickr30Kannotation r@1: 43.9 -> 46.7Improved
add cross-entropy losses alongside semantic centers (ablation)joint embedding · Flickr8K/Flickr30Kannotation r@1: 30.1 -> 32.3No clear effect
add quantized center loss (1000 centers) (ablation)ResNet152 embedding · Flickr30K, no random cropsannotation r@1: 52.9 -> 57.2Improved
quantize the center loss instead of unquantized (ablation)ResNet152 embedding · Flickr30K, no random cropsannotation r@1: 55.8 -> 57.2Improved
add center loss with random crops (ablation)ResNet152 embedding · Flickr30K, random cropsannotation r@1: 57.1 -> 60.1Improved
add center loss on VSRN backbone (ablation)VSRN embedding · Flickr30Kannotation r@1: 68.1 -> 69.8Improved
vary number of semantic centers (ablation)ResNet152 embedding · Flickr30K and MS-COCO 1Kaccuracy: maximal difference 0.1%No clear effect
replace learnt embedding with bert-large-uncased (ablation)VSRN embedding · Flickr30Kannotation r@1: 69.8 -> 70.6Improved
add adaptive margin c=1.03 (ablation)triplet loss · Flickr30Kaverage recall: 70.1 -> 70.7Improved
use adaptive margin of Wu instead of proposed (ablation)triplet loss · Flickr30Kaverage recall: 70.7 -> 44.6Hurt
use adaptive margin of Li instead of proposed (ablation)triplet loss · Flickr30Kaverage recall: 70.7 -> 55.9Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy