AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Efficient Inference of Vision Instruction-Following Models with Elastic Cache

Zuyan Liu, Benlin Liu, Jiahui Wang, Yuhao Dong, Guangyi Chen, Yongming Rao, Ranjay Krishna, Jiwen Lu

ECCV 2024 · Vision-language and multimodal › Multimodal LLMs · approach: Attention-based visual token pruning · 25 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use fixed-point discard during generation instead of discarding the most recent cache (ablation)Elastic Cache · LLaVA-Description, budget 0.5PPL: 3.93 -> 3.60Improved
use fixed-point discard instead of frequency policy during generation (ablation)Elastic Cache · LLaVA-Description, budget 0.5PPL: 3.75 -> 3.60Improved
merge less-important cache into anchors instead of evicting (ablation)Elastic Cache · LLaVA-Description, budget 0.5PPL: 3.68 -> 3.60Improved
use simple bucket averaging instead of clustering for merging (ablation)Elastic Cache · LLaVA-Description, budget 0.5PPL: 3.61 -> 3.60No clear effect
use layer-wise importance rather than head-wise or shared (ablation)Elastic Cache · LLaVA-Description, budget 0.5PPL: 3.75 -> 3.60Improved
use sum of attention scores as importance instead of mean or moving average (ablation)Elastic Cache · LLaVA-Description, budget 0.5PPL: 8.70 -> 3.60Improved
replace Local cache with Elastic Cache at budget 0.5LLaVA-1.5-13B · LLaVA-DescriptionPPL: 28.72 lower PPLImproved
replace H2O with Elastic Cache at budget 0.5LLaVA-1.5-13B · LLaVA-DescriptionPPL: 4.34 lower PPLImproved
apply Elastic Cache at budget 0.2 versus full cacheLLaVA-1.5-7B · batch 16, 1024+512 tokensthroughput (token/s): 267.7 -> 476.3Improved
increase pruning ratio to 0.3 (budget 0.3) for Local cacheLocal cache · GPT-4V win ratewin rate (%): 46.37 -> 10.10Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy