AI Paper Atlas

Research map / Detection and segmentation / Open-set and few-shot detection

A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-Language Model

Mengde Xu, Zheng Zhang, Fangyun Wei, Yutong Lin, Yue Cao, Han Hu, Xiang Bai

ECCV 2022 · Detection and segmentation › Open-set and few-shot detection · approach: Open-vocabulary semantic segmentation · 330 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use two-stage mask-proposal + CLIP instead of FCNCLIP segmentation · Cityscapes cross-datasetmIoU: 21.4 -> 34.5Improved
ensemble CLIP vision encoder with retrained encoder (ablation)two-stage framework · COCO Stuff zero-shothIoU: 28.2 -> 37.8Improved
use MaskFormer proposals instead of Selective Search (ablation)two-stage framework · COCO Stuff zero-shothIoU: 11.0 -> 28.2Improved
use MaskFormer proposals instead of GPB-UCM (ablation)two-stage framework · COCO Stuff zero-shothIoU: 10.9 -> 28.2Improved
use learnable prompt instead of manual prompt (ablation)two-stage framework · COCO Stuff zero-shothIoU: +9.9 hIoUImproved
use CLIP ViT-B/16 instead of ResNet-101 (ablation)region classifier · COCO Stuff zero-shot (manual prompt)hIoU: 13.8 -> 18.3Improved
add sliding window testing to FCN (ablation)FCN CLIP baseline · COCO Stuff zero-shothIoU: 11.7 -> 20.9Improved
use CLIP-VL pretraining instead of ImageNet (ablation)two-stage framework ResNet-101 · Pascal VOC 2012 zero-shothIoU: 49.5 -> 74.2Improved
add self-training (ablation)two-stage framework · COCO Stuff zero-shothIoU: 37.8 -> 41.5Improved
fill background with learnable pixel prompts or mask token instead of mean values (ablation)image prompt engineering · COCO Stuff zero-shothIoU: 18.3 -> FailedHurt
fill background with mean values instead of preserving context pixels (ablation)image prompt engineering · COCO Stuff zero-shothIoU: 9.3 -> 18.3Improved
use all samples per category instead of 32 for prompt learning (ablation)learnable prompt (length 32) · COCO Stuff zero-shotUnseen Acc: 32.8 -> 27.6Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy