AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers

Dachuan Shi, Chaofan Tao, Ying Jin, Zhendong Yang, Chun Yuan, Jiaqi Wang

ICML 2023 · Vision-language and multimodal › Cross-modal learning · approach: Multi-modality image fusion · 6 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Unified search across modalities and structures (vs manual per-component ratios) (ablation)BLIP compression, NLVR2 · NLVR2 2xTest Acc: +3.88 over mask-based pruning (80.32 vs 76.44)Improved
Unified search instead of manual per-component ratios (ablation)BLIP compression, NLVR2 · NLVR2 3xTest Acc: Mask-based fails to converge; unified reaches 71.66Improved
Progressive pruning added to unified search (ablation)BLIP compression, NLVR2 · NLVR2 2xTest Acc: +0.81 (81.13 vs 80.32)Improved
Progressive pruning added to unified search (ablation)BLIP compression, NLVR2 · NLVR2 3xTest Acc: unified 71.66 -> UPop 77.61 (+5.95)Improved
Progressive pruning added to unified search (ablation)BLIP compression, NLVR2 · NLVR2 4x-10xconvergence: UPop converges at 4x, 5x, 10x; unified and mask-based fail at 4xImproved
Progressive pruning (search with no retraining) (ablation)BLIP 2x, NLVR2 · NLVR2 no retrainconvergence: UPop reaches 77.84 test; unified and mask-based do not convergeImproved
Progressive pruning with one retrain epoch (ablation)BLIP 2x, NLVR2 · NLVR2 one retrain epochTest Acc: UPop 80.08 vs unified 75.30 vs mask-based 63.35Improved
Adaptive unified ratio assignment vs manual assignment (ablation)BLIP 2x, NLVR2 · NLVR2 2xTest Acc: adaptive 81.13 vs manual range 61.31 to 80.01Improved
Unified + progressive UPop vs mask-based pruningBLIP retrieval COCO and captioning/VQA · COCO retrieval, Flickr30K retrieval, VQA, Caption 2xR@1 / accuracy / CIDEr: e.g. COCO I->T R@1 77.4 vs 61.7; VQA 76.3 vs 71.6Improved
Unified + progressive vs mask-based at 4xBLIP retrieval COCO · COCO 4xR@1 (Image->Text): UPop 62.9 vs mask-based not convergedImproved
Progressive pruning added to unified search (ablation)BLIP retrieval COCO · COCO 4xR@1 (Image->Text): 40.3 -> 62.9Improved
Replace progressive pruning with iterative pruning with STE (ablation)BLIP NLVR2 2x · NLVR2 2xTest Acc: 80.54 (iterative STE) vs 81.13 (progressive)Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy