AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Accelerating Vision-Language Pretraining With Free Language Modeling

Teng Wang, Yixiao Ge, Feng Zheng, Ran Cheng, Ying Shan, Xiaohu Qie, Ping Luo

CVPR 2023 · Vision-language and multimodal › Human and embodied modeling · approach: Vision-language pretraining · 9 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace MLM with FLM using 100% prediction and flexible span corruptionvision-language pretraining with CLIP-B/32 and RoBERTa · Pretraining convergenceGPU Days: 58.7 -> 22.7; 2.5×Improved
replace MLM with FLMvision-language pretraining with CLIP-B/32 and RoBERTa · NLVR² testaccuracy: 78.60 -> 78.63Improved
replace MLM with FLMvision-language pretraining with CLIP-B/32 and RoBERTa · Flickr30K IR@1IR@1: 73.41 -> 72.81Hurt
replace MLM with FLMvision-language pretraining with CLIP-B/32 and RoBERTa · COCO CaptioningCIDEr: 120.3 -> 123.0Improved
add intermediate bidirectional prediction losses (ablation)FLM · VQA test-devaccuracy: 73.04 -> 73.85Improved
add left-to-right rather than right-to-left intermediate prediction loss (ablation)FLM · VQA test-devaccuracy: 73.38 -> 73.67Improved
share parameters between forward and reverse text encoders (ablation)FLM · NLVR² testaccuracy: 77.51 -> 78.63Improved
increase prediction rate from 50% to 100% (ablation)FLM · NLVR² testaccuracy: 77.47 -> 78.63Improved
increase span corruption rate from 1/L to 30% (ablation)FLM · NLVR² testaccuracy: 78.63 -> 78.83Improved
increase span corruption rate from 1/L to 50% (ablation)FLM · NLVR² testaccuracy: 78.63 -> 77.84Hurt
use random rather than span corruption at 30% (ablation)FLM · NLVR² testaccuracy: 78.63 -> 77.74Hurt
use a deeper reconstructor with 6 bottom and 6 top layers instead of 1 top layer (ablation)FLM · NLVR² testaccuracy: 77.49 -> 78.63Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy