Research map / Detection and segmentation / Open-set and few-shot detection
Open-Vocabulary Semantic Segmentation With Mask-Adapted CLIP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt CLIP with diverse caption nouns paired to generated mask proposals instead of only closed-set segmentation labels (ablation) | OVSeg · A-847 | mIoU: 5.3 -> 8.8 | Helped |
| Adapt CLIP with diverse caption nouns paired to generated mask proposals instead of only closed-set segmentation labels (ablation) | OVSeg · A-150 | mIoU: 23.0 -> 28.8 | Helped |
| Use ground-truth masks and fixed ground-truth classes for CLIP adaptation (ablation) | OVSeg · A-847 | mIoU: -2.0 | Hurt |
| Use generated proposals rather than ground-truth masks with caption nouns (ablation) | OVSeg · A-150 | mIoU: 24.2 -> 28.8 | Helped |
| Use five captions per image rather than one (ablation) | OVSeg · A-150 | mIoU: -0.2% | Hurt |
| Add mask prompt tuning without full CLIP finetuning (ablation) | OVSeg · A-150 | mIoU: +4.7% | Helped |
| Add mask prompt tuning after full CLIP finetuning (ablation) | OVSeg · A-150 | mIoU: 28.8 -> 29.6; +0.8% | Helped |
| Apply mask prompt tuning before full finetuning rather than after it (ablation) | OVSeg · A-150 | mIoU: -1.5% | Hurt |
| Optimize mask prompts and CLIP weights simultaneously rather than applying prompts after finetuning (ablation) | OVSeg · A-150 | mIoU: -0.6% | Hurt |
| Use prompt depth 3 rather than depth 1 (ablation) | Mask prompt tuning · A-150 without full finetuning | mIoU: 25.7 -> 26.5 | Helped |
| Increase prompt depth from 3 to 6 or 12 (ablation) | Mask prompt tuning · A-150 with full finetuning | mIoU: 29.6 -> 29.4; 29.6 -> 29.3 | Hurt |
| Replace masked prompts with added deep visual prompt tokens (ablation) | CLIP prompt adaptation · A-150 | mIoU: 26.5 -> 25.5 | Hurt |
Closest papers by meaning
- Mask-Adapter: The Devil is in the Masks for Open-Vocabulary Segmentation (CVPR 2025 · similarity 0.94)
- Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models (CVPR 2024 · similarity 0.93)
- CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor (CVPR 2024 · similarity 0.92)
- Dual Semantic Guidance for Open Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.92)
- Unveiling the Knowledge of CLIP for Training-Free Open-Vocabulary Semantic Segmentation (AAAI 2025 · similarity 0.92)
- Effective SAM Combination for Open-Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.