Research map / Detection and segmentation / Open-set and few-shot detection
SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the reconstruction loss (ablation) | SegCLIP without superpixel KL loss · PASCAL VOC 2012, PASCAL Context, and COCO val | mIoU: +1.19%, +0.92%, and +0.66% | Helped |
| Add the reconstruction loss (ablation) | SegCLIP with superpixel KL loss · PASCAL VOC 2012, PASCAL Context, and COCO val | mIoU: +4.11%, +0.56%, and +0.52% | Helped |
| Add the superpixel-based KL loss (ablation) | SegCLIP without reconstruction loss · PASCAL VOC 2012, PASCAL Context, and COCO val | mIoU: +0.54%, +0.72%, and +1.07% | Helped |
| Add the superpixel-based KL loss (ablation) | SegCLIP with reconstruction loss · PASCAL VOC 2012, PASCAL Context, and COCO val | mIoU: +3.46%, +0.36%, and +0.93% | Helped |
| Move the semantic group module from layer 10 to layer 6 or 8 (ablation) | SegCLIP · PASCAL VOC 2012 val | mIoU: 47.95 -> 35.28; 47.95 -> 43.75 | Hurt |
| Move the semantic group module from layer 10 to layer 11 (ablation) | SegCLIP · PASCAL VOC 2012 val | mIoU: 47.95 -> 22.07 | Hurt |
| Use 6, 8, or 10 learnable centers instead of the default 8 (ablation) | SegCLIP with the group module at layer 10 · PASCAL VOC 2012, PASCAL Context, and COCO val | mIoU: VOC: 43.75, 47.03, 47.95, 44.89; Context: 22.71, 23.36, 23.43, 23.46; COCO: 21.40, 24.85, 24.86, 24.74 | Mixed |
| Add cross-attention to the semantic group module (ablation) | SegCLIP with no cross-attention layers · PASCAL VOC 2012, PASCAL Context, and COCO val | mIoU: VOC: 44.44 -> 47.63; Context: 22.28 -> 23.29; COCO: 22.11 -> 24.21 | Helped |
| Use 4 cross-attention layers instead of 2 (ablation) | SegCLIP · PASCAL VOC 2012, PASCAL Context, and COCO val | mIoU: VOC: 47.95 -> 45.39; Context: 23.43 -> 23.17; COCO: 24.86 -> 23.80 | Hurt |
| Initialize SegCLIP with pretrained CLIP weights rather than train from scratch | SegCLIP · PASCAL VOC 2012, PASCAL Context, and COCO val | mIoU: 19.3%, 5.6%, and 11.3% | Helped |
| Use a patch size of 32 instead of 16 (ablation) | SegCLIP · PASCAL VOC 2012, PASCAL Context, and COCO val | mIoU: VOC: 52.5 -> 44.2; Context: 24.7 -> 22.0; COCO: 26.5 -> 21.4 | Hurt |
| Train SegCLIP rather than GroupViT 1-stage on CC+COCO | SegCLIP trained from scratch · PASCAL VOC 2012, PASCAL Context, and COCO val | mIoU: +5.2%, +4.3%, and +2.3% | Helped |
Closest papers by meaning
- Exploring Simple Open-Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.91)
- CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation (CVPR 2024 · similarity 0.91)
- Seg-ReSearch: Segmentation with Interleaved Reasoning and External Search (ICML 2026 · similarity 0.89)
- SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentation (CVPR 2024 · similarity 0.88)
- Dual Semantic Guidance for Open Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.88)
- Unveiling the Knowledge of CLIP for Training-Free Open-Vocabulary Semantic Segmentation (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.