AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

Perspectives and Prospects on Transformer Architecture for Cross-Modal Tasks with Language and Vision

Andrew Shin, Masato Ishii, Takuya Narihira

IJCV 2022 · Vision-language and multimodal › Cross-modal learning · approach: Contrastive vision-language alignment · 58 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add large-scale pre-training (ablation)ViLBERT · VQA (dev)accuracy: 68.93 -> 70.55Improved
add large-scale pre-training (ablation)VL-BERT · VQA (dev)accuracy: 69.58 -> 71.16Improved
add pre-training (ablation)InterBERT · VCR (Q/A)accuracy: 63.6 -> 73.1Improved
add pre-training (ablation)Unicoder-VL · Image retrievalR@1: 57.8 -> 71.5Improved
add pre-training (ablation)ViLBERT · Image retrievalR@1: 45.50 -> 58.20Improved
scale model from base to largeVL-BERT · VQA (dev)accuracy: 71.16 -> 71.79Improved
scale model from base to largeUNITER · VQA (dev)accuracy: 72.70 -> 73.82Improved
scale image backbone from ResNet-50 to ResNeXt-152PixelBERT · VQA (dev)accuracy: 71.35 -> 74.45Improved
add Visual Genome to COCO pre-training dataLAMP · VQA (dev)accuracy: 70.85 -> 72.48Improved
pre-train on video-text with YouCook2 annotations (UniVL) instead of YouTube-only (VideoBERT)video-language transformers · YouCook2CIDEr: 0.55 -> 1.81Improved
add dataset-matched pre-training for target task (image QA data)LXMERT, OSCAR, SemVLP · VQA (dev)accuracyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy