AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Explainable Semantic Space by Grounding Language to Vision with Cross-Modal Contrastive Learning

Yizhen Zhang, Minkyu Choi, Kuan Han, Zhongming Liu

NeurIPS 2021 · Vision-language and multimodal › Human and embodied modeling · approach: Language-grounded visual concept learning · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add self-attention to the visual streamVGG-based visual stream · ImageNet validationtop-1 accuracy: 71.6% -> 74.3%Improved
ground language representations in visionBert language representations · Human-rated concreteness, word categoriesPearson's r: 0.3538 -> 0.8749Improved
ground language representations in visionBert language representations · Human-rated concreteness, individual wordsPearson's r: 0.1040 -> 0.6615Improved
also ground object relationsGrounded language representations · Human-rated concreteness, word categoriesPearson's r: 0.8749 -> 0.8001Hurt
also ground object relationsGrounded language representations · Human-rated concreteness, individual wordsPearson's r: 0.6615 -> 0.6948Improved
ground language representations in visionBert language representations · striped horse → zebracosine similarity: 0.12 -> 0.60Improved
also ground object relationsGrounded language representations · striped horse → zebracosine similarity: 0.60 -> 0.63Improved
ground language representations in visionBert language representations · black and white bear → pandacosine similarity: 0.13 -> 0.69Improved
ground language representations in visionBert language representations · red fruit → strawberrycosine similarity: 0.39 -> 0.75Improved
ground language representations in visionBert language representations · young dog → puppycosine similarity: 0.40 -> 0.92Improved
fine-tune more of the higher Bert layers during image-text grounding (ablation)Bert language stream · MS COCO image-text retrievalimage-text retrieval performanceImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy