Research map / Vision-language and multimodal / Human and embodied modeling
Accelerating Vision-Language Pretraining With Free Language Modeling
Teng Wang, Yixiao Ge, Feng Zheng, Ran Cheng, Ying Shan, Xiaohu Qie, Ping Luo
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace MLM with FLM using 100% prediction and flexible span corruption | vision-language pretraining with CLIP-B/32 and RoBERTa · Pretraining convergence | GPU Days: 58.7 -> 22.7; 2.5× | Improved |
| replace MLM with FLM | vision-language pretraining with CLIP-B/32 and RoBERTa · NLVR² test | accuracy: 78.60 -> 78.63 | Improved |
| replace MLM with FLM | vision-language pretraining with CLIP-B/32 and RoBERTa · Flickr30K IR@1 | IR@1: 73.41 -> 72.81 | Hurt |
| replace MLM with FLM | vision-language pretraining with CLIP-B/32 and RoBERTa · COCO Captioning | CIDEr: 120.3 -> 123.0 | Improved |
| add intermediate bidirectional prediction losses (ablation) | FLM · VQA test-dev | accuracy: 73.04 -> 73.85 | Improved |
| add left-to-right rather than right-to-left intermediate prediction loss (ablation) | FLM · VQA test-dev | accuracy: 73.38 -> 73.67 | Improved |
| share parameters between forward and reverse text encoders (ablation) | FLM · NLVR² test | accuracy: 77.51 -> 78.63 | Improved |
| increase prediction rate from 50% to 100% (ablation) | FLM · NLVR² test | accuracy: 77.47 -> 78.63 | Improved |
| increase span corruption rate from 1/L to 30% (ablation) | FLM · NLVR² test | accuracy: 78.63 -> 78.83 | Improved |
| increase span corruption rate from 1/L to 50% (ablation) | FLM · NLVR² test | accuracy: 78.63 -> 77.84 | Hurt |
| use random rather than span corruption at 30% (ablation) | FLM · NLVR² test | accuracy: 78.63 -> 77.74 | Hurt |
| use a deeper reconstructor with 6 bottom and 6 top layers instead of 1 top layer (ablation) | FLM · NLVR² test | accuracy: 77.49 -> 78.63 | Improved |
Closest papers by meaning
- Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models (NeurIPS 2025 · similarity 0.88)
- VILA: On Pre-training for Visual Language Models (CVPR 2024 · similarity 0.88)
- LLMC+: Benchmarking Vision-Language Model Compression with a plug-and-play Toolkit (AAAI 2026 · similarity 0.87)
- MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models (NeurIPS 2024 · similarity 0.87)
- VL-Cache: Sparsity and Modality-Aware KV Cache Compression for Vision-Language Model Inference Acceleration (ICLR 2025 · similarity 0.87)
- Unveiling Encoder-Free Vision-Language Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
