AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

FALIP: Visual Prompt as Foveal Attention Boosts CLIP Zero-Shot Performance

Jiedong Zhuang, Jiaqi Hu, Lianrui Mu, Rui Hu, Xiaoyu Liang, Jiangnan Ye, Haoji Hu

ECCV 2024 · Vision-language and multimodal › CLIP and zero-shot · approach: Attribute-based zero-shot learning · 9 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
insert foveal attention mask on the [CLS] row (Method a) instead of all rows (ablation)FALIP · REC average (RefCOCO family)average accuracy: 39.3 -> 45.2Improved
insert foveal attention mask on the [CLS] row instead of the diagonal (ablation)FALIP · REC average (RefCOCO family)average accuracy: 15.8 -> 45.2Improved
use attention mask instead of feature mask (ablation)FALIP · REC average (RefCOCO family)average accuracy: 27.0 -> 45.2Improved
add foveal attention mask to original CLIP (ablation)CLIP ViT-B/16 · REC average (RefCOCO family)average accuracy: 21.5 -> 45.2Improved
apply RedCircle visual prompt to CLIP for fine-grained classificationCLIP ViT-B/16 · CUB-200-2011Top1: 54.2 -> 44.2Hurt
apply Blur visual prompt to CLIPCLIP ViT-B/16 · WaterbirdsTop1: 78.2 -> 78.1No clear effect
use FALIP instead of original CLIP for classificationCLIP ViT-B/16 · CUB-200-2011Top1: 54.2 -> 54.3No clear effect
use FALIP instead of Blur on Waterbirds with ViT-LCLIP ViT-L/14 · Waterbirds (ViT-L)Top1: 85.1 -> 84.5Hurt
unleash salient attention heads in last 4 layers for RedCircle (ablation)RedCircle · REC average (RefCOCO family)average accuracy: 41.3 -> 45.2Improved
insert mask in layers 9 to 12 instead of layers 1 to 4 (ablation)FALIP · REC average (RefCOCO family)average accuracy: 24.0 -> 45.2Improved
use alpha=0.2 and sigma=100 instead of sigma=1 (ablation)FALIP · REC average (RefCOCO family)average accuracy: 39.5 -> 45.2Improved
add FALIP on top of RedCircle with ensemble and post-processingRedCircle · RefCOCO TestA (gold, with E and P)accuracy: 53.5 -> 53.4Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy