AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

BUS: Efficient and Effective Vision-Language Pre-Training with Bottom-Up Patch Summarization.

Chaoya Jiang, Haiyang Xu, Wei Ye, Qinghao Ye, Chenliang Li, Ming Yan, Bin Bi, Shikun Zhang, Fei Huang, Songfang Huang

ICCV 2023 · Vision-language and multimodal › Human and embodied modeling · approach: Vision-language pretraining · 9 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add bottom-up patch summarization (KPE + TPA) to the VLP model (ablation)BUS · VQA test-devaccuracy / throughput: 77.58 -> 77.89; throughput 170.23 -> 360.23Improved
add key patch extraction in ViT backbone (ablation)BUS · VQA test-devthroughput: 246.56 -> 360.23Improved
add text-guided patch abstraction decoder (ablation)BUS · VQA test-devaccuracy: 77.64 -> 77.89Improved
add text alignment score for patch selection (ablation)BUS · VQA test-devaccuracy: 77.43 -> 77.89Improved
add image [CLS] attention map to patch scoring (ablation)BUS · VQA test-devaccuracy: 77.67 -> 77.89Improved
place selector after layer 4 with 40% keep ratio instead of layer 6 with 70% (ablation)BUS · VQA test-devaccuracy: 77.89 -> 76.22Hurt
increase KPE keep ratio from 70% to 90% at layer 6 (ablation)BUS · VQA test-devaccuracy: 77.89 -> 77.92No clear effect
raise input image resolution from 384 to 512 at similar compute (ablation)BUS · VQA test-devaccuracy: 77.89 -> 78.28Improved
use BUS instead of mPLUG baseline at 384 resolutionBUS · VQA test-devFLOPs (G): 84.87 -> 47.24Improved
increase TPA selection ratio beyond 20%BUS · VQA test-devaccuracyMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy