AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

Universal Multimodal Representation for Language Understanding

Zhuosheng Zhang, Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, Zuchao Li, Hai Zhao

TPAMI 2023 · Vision-language and multimodal › CLIP and zero-shot · approach: Recaptioning and text-supervised pretraining · 39 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add UVR-TILT image retrievalTransformer-Base · WMT16 En-RoBLEU: 32.66 -> 33.78Improved
add UVR-CMRM image retrieval (COCO)Transformer-Big · WMT14 En-DeBLEU: 28.45 -> 29.21Improved
add UVR-TILT to BERT-WWMBERT-WWM · GLUE RTE testaccuracy: 77.3 -> 84.5Improved
add UVR-TILT to BERT-WWMBERT-WWM · CoLAMCC: 63.6 -> 62.5Hurt
add UVR-TILT to MT-DNN largeMT-DNN large · GLUE RTE validationaccuracy: 78.70 -> 82.19Improved
add UVR-TILT to MT-DNN baseMT-DNN base · GLUE QNLI validationaccuracy: 90.32 -> 91.31Improved
shuffle image features but keep lookup table (ablation)UVR-TILT · WMT16 En-RoBLEU: 33.72 -> 33.40Improved
keep trained image features instead of random init (ablation)UVR-TILT · WMT16 En-RoBLEU: 33.08 -> 33.72Improved
keep sentence-image mapping instead of random mapping (ablation)UVR-TILT · WMT16 En-RoBLEU: 32.05 -> 33.72Improved
use ResNet152 instead of ResNet50 features (ablation)UVR-TILT · WMT16 En-RoBLEU: 33.72 -> 33.82No clear effect
add UVR-TILT when source content words are removedTransformer · Multi30K En-De test2016BLEU: 10.94 -> 12.80Improved
add UVR-CMRMTransformer · WAT'19 En-Hi challengeBLEU: 20.52 -> 21.62Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy