Research map / Detection and segmentation / Open-set and few-shot detection
A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-Language Model
Mengde Xu, Zheng Zhang, Fangyun Wei, Yutong Lin, Yue Cao, Han Hu, Xiang Bai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use two-stage mask-proposal + CLIP instead of FCN | CLIP segmentation · Cityscapes cross-dataset | mIoU: 21.4 -> 34.5 | Improved |
| ensemble CLIP vision encoder with retrained encoder (ablation) | two-stage framework · COCO Stuff zero-shot | hIoU: 28.2 -> 37.8 | Improved |
| use MaskFormer proposals instead of Selective Search (ablation) | two-stage framework · COCO Stuff zero-shot | hIoU: 11.0 -> 28.2 | Improved |
| use MaskFormer proposals instead of GPB-UCM (ablation) | two-stage framework · COCO Stuff zero-shot | hIoU: 10.9 -> 28.2 | Improved |
| use learnable prompt instead of manual prompt (ablation) | two-stage framework · COCO Stuff zero-shot | hIoU: +9.9 hIoU | Improved |
| use CLIP ViT-B/16 instead of ResNet-101 (ablation) | region classifier · COCO Stuff zero-shot (manual prompt) | hIoU: 13.8 -> 18.3 | Improved |
| add sliding window testing to FCN (ablation) | FCN CLIP baseline · COCO Stuff zero-shot | hIoU: 11.7 -> 20.9 | Improved |
| use CLIP-VL pretraining instead of ImageNet (ablation) | two-stage framework ResNet-101 · Pascal VOC 2012 zero-shot | hIoU: 49.5 -> 74.2 | Improved |
| add self-training (ablation) | two-stage framework · COCO Stuff zero-shot | hIoU: 37.8 -> 41.5 | Improved |
| fill background with learnable pixel prompts or mask token instead of mean values (ablation) | image prompt engineering · COCO Stuff zero-shot | hIoU: 18.3 -> Failed | Hurt |
| fill background with mean values instead of preserving context pixels (ablation) | image prompt engineering · COCO Stuff zero-shot | hIoU: 9.3 -> 18.3 | Improved |
| use all samples per category instead of 32 for prompt learning (ablation) | learnable prompt (length 32) · COCO Stuff zero-shot | Unseen Acc: 32.8 -> 27.6 | Hurt |
Closest papers by meaning
- Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models (CVPR 2024 · similarity 0.93)
- Dual Semantic Guidance for Open Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.92)
- Towards Open-Vocabulary Semantic Segmentation Without Semantic Labels (NeurIPS 2024 · similarity 0.92)
- Effective SAM Combination for Open-Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.92)
- Training-free Open-Vocabulary Semantic Segmentation via Diverse Prototype Construction and Sub-region Matching (AAAI 2025 · similarity 0.92)
- CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor (CVPR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
