AI Paper Atlas

Research map / Detection and segmentation / Open-set and few-shot detection

Explore the Potential of CLIP for Training-Free Open Vocabulary Semantic Segmentation

Tong Shao, Zhuotao Tian, Hang Zhao, Jingyong Su

ECCV 2024 · Detection and segmentation › Open-set and few-shot detection · approach: Open-vocabulary semantic segmentation · 40 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add semantic correlation recovery (self-correlation attention in last layer) to CLIP (ablation)CLIP ViT-B/16 · Avg. of 4 benchmarks (224)mIoU: 14.47 -> 36.94Improved
add patch clustering with DBSCAN on attention weights (ablation)CLIPtrase (SCR) · Avg. of 4 benchmarks (224)mIoU: 36.94 -> 38.02Improved
add denoising to discard global-patch noise clusters (ablation)CLIPtrase (SCR+PC) · Avg. of 4 benchmarks (224)mIoU: 38.02 -> 38.50Improved
use cosine similarity instead of inner product for correlation (ablation)CLIPtrase · Avg. of 4 benchmarks (224)mIoU: 25.91 -> 38.50Improved
cluster attention weight prototypes rather than features (ablation)CLIPtrase · Avg. of 4 benchmarks (224)mIoU: 35.61 -> 38.50Improved
apply correlation recovery at last layer rather than layers 9-11 (ablation)CLIPtrase · Avg. of 4 benchmarks (224)mIoU: 17.11 (layer 11) -> 38.50 (layer 12)Improved
increase input resolution from 224 to 336 (ablation)CLIPtrase · Avg. of 4 benchmarksmIoU: 38.50 -> 39.31Improved
replace patch clustering masks with SAM using CLIPtrase point promptsCLIPtrase · Avg. of 9 benchmarksmIoU: 33.54 -> 34.49Improved
restore correlation within ground-truth regions (oracle) vs cosine over all patches (ablation)CLIPtrase SCR · Avg. of VOC, ADE, COCO, PC (224)mIoU: 36.94 -> 37.67Improved
use DINOv2 for masking instead of CLIPtrase clustering (ablation)CLIPtrase · Avg. of VOC, ADE, COCO, PC (224)mIoU: 38.50 -> 37.59Hurt
initialize SAN query embeddings with CLIPtrase features and masksSAN · SAN training convergenceconvergence speedImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy