Research map / Vision-language and multimodal / Human and embodied modeling
BUS: Efficient and Effective Vision-Language Pre-Training with Bottom-Up Patch Summarization.
Chaoya Jiang, Haiyang Xu, Wei Ye, Qinghao Ye, Chenliang Li, Ming Yan, Bin Bi, Shikun Zhang, Fei Huang, Songfang Huang
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add bottom-up patch summarization (KPE + TPA) to the VLP model (ablation) | BUS · VQA test-dev | accuracy / throughput: 77.58 -> 77.89; throughput 170.23 -> 360.23 | Improved |
| add key patch extraction in ViT backbone (ablation) | BUS · VQA test-dev | throughput: 246.56 -> 360.23 | Improved |
| add text-guided patch abstraction decoder (ablation) | BUS · VQA test-dev | accuracy: 77.64 -> 77.89 | Improved |
| add text alignment score for patch selection (ablation) | BUS · VQA test-dev | accuracy: 77.43 -> 77.89 | Improved |
| add image [CLS] attention map to patch scoring (ablation) | BUS · VQA test-dev | accuracy: 77.67 -> 77.89 | Improved |
| place selector after layer 4 with 40% keep ratio instead of layer 6 with 70% (ablation) | BUS · VQA test-dev | accuracy: 77.89 -> 76.22 | Hurt |
| increase KPE keep ratio from 70% to 90% at layer 6 (ablation) | BUS · VQA test-dev | accuracy: 77.89 -> 77.92 | No clear effect |
| raise input image resolution from 384 to 512 at similar compute (ablation) | BUS · VQA test-dev | accuracy: 77.89 -> 78.28 | Improved |
| use BUS instead of mPLUG baseline at 384 resolution | BUS · VQA test-dev | FLOPs (G): 84.87 -> 47.24 | Improved |
| increase TPA selection ratio beyond 20% | BUS · VQA test-dev | accuracy | Mixed |
Closest papers by meaning
- TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment (CVPR 2026 · similarity 0.89)
- SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference (ICML 2025 · similarity 0.88)
- Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs (ICLR 2026 · similarity 0.88)
- Probing Visual Language Priors in VLMs (ICML 2025 · similarity 0.88)
- Modality-Specialized Synergizers for Interleaved Vision-Language Generalists (ICLR 2025 · similarity 0.87)
- DUET-VLM: Dual stage Unified Efficient Token reduction for VLM Training and Inference (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
