AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

MACK: Multimodal Aligned Conceptual Knowledge for Unpaired Image-text Matching

Yan Huang, Yuming Wang, Yunan Zeng, Liang Wang

NeurIPS 2022 · Vision-language and multimodal › Human and embodied modeling · approach: Language-grounded scene text and structure · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace prototypical region representation with random region representation (ablation)pretrained general knowledge · Flickr30kRs: 106.2 -> 31.7Hurt
remove adjectives when recomputing word representations (ablation)pretrained general knowledge · Flickr30kRs: 106.2 -> 101.5Hurt
replace max-mean pooling with mean-mean pooling (ablation)pretrained general knowledge · Flickr30kRs: 106.2 -> 62.2Hurt
add fine-tuning on unpaired images and texts with cycle-consistent lossMACK · Flickr30kRs: 106.2 -> 153.5Improved
remove implicit one-to-one prior (ablation)fine-tuned domain knowledge · Flickr30kRs: 153.5 -> 151.1Hurt
remove knowledge as cross-modal bridge (ablation)fine-tuned domain knowledge · Flickr30kRs: 153.5 -> 3.2Hurt
re-rank CLIP with MACKCLIP zero-shot · Flickr30k and MSCOCORs: 525.6 -> 530.0; 386.1 -> 392.8Improved
re-rank ALBEF with MACKALBEF zero-shot · Flickr30k and MSCOCORs: 506.9 -> 513.3; 428.3 -> 430.9Improved
re-rank VSRN with MACKVSRN cross-dataset · MSCOCO -> Flickr30k, Flickr30k -> MSCOCORs: 409.9 -> 411.7; 198.4 -> 201.4Improved
re-rank SAEM with MACKSAEM cross-dataset · MSCOCO -> Flickr30k, Flickr30k -> MSCOCORs: 415.5 -> 417.9; 219.8 -> 223.4Improved
raise max regions per prototype f from 20 to infinity (ablation)ALBEF + MACK · Flickr30kRs: 509.8 -> 513.3Improved
raise max regions per prototype f from 20 to infinity (ablation)ALBEF + MACK · MSCOCORs: 427.5 -> 430.9Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy