Research map / Detection and segmentation / Open-set and few-shot detection
CLIP-Adapted Region-to-Text Learning for Generative Open-Vocabulary Semantic Segmentation
Jiannan Ge, Lingxi Xie, Hongtao Xie, Pandeng Li, Sun-Ao Liu, Xiaopeng Zhang, Qi Tian, Yongdong Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the language model rather than keeping it frozen (ablation) | CRTNet · P-59 / A-150 / P-459 / A-847 | mIoU: 47.5 -> 49.9 (P-59), 20.2 -> 23.6 (A-150), 12.3 -> 15.1 (P-459), 5.2 -> 7.9 (A-847) | Improved |
| add CC3M-595K to VG and COCO training data (ablation) | CRTNet · ADE-847 | mIoU: 5.4 -> 7.9 | Improved |
| add COCO to VG training data (ablation) | CRTNet · ADE-150 | mIoU: 21.4 -> 21.2 (VG alone vs VG+COCO) | Hurt |
| add VG and COCO region-level data to CC3M-only training (ablation) | CRTNet · VOC | mIoU: 84.7 -> 91.4 | Improved |
| use temperature temp=5 instead of 0.5 (ablation) | text-aware feature aggregation · VOC | mIoU: 78.4 -> 85.2 | Improved |
| raise temp from 5 to 100 (ablation) | text-aware feature aggregation · VOC | mIoU: 85.2 -> 83.4 | Hurt |
| filter 60% of noun phrases from CC3M captions instead of none (ablation) | data engine · P-459 | mIoU: 7.9 -> 9.0 | Improved |
| filter 60% of noun phrases instead of none (ablation) | data engine · A-150 | mIoU: 16.8 -> 18.7 | Improved |
| increase beam size from 1 to 3 (ablation) | CRTNet inference · A-150 | mIoU: 22.3 -> 23.6 | Improved |
| increase beam size from 3 to 5 (ablation) | CRTNet inference · A-150 | mIoU: 23.6 -> 23.1 | Hurt |
| raise CC3M proportion from 1:1:1 to 4:1:1 (ablation) | training data mix · A-150 | mIoU: 23.6 -> 22.5 | Hurt |
| use FlanT5-Base encoder-decoder instead of OPT-350M (ablation) | language model · VOC | mIoU: 79.2 -> 90.7 | Improved |
Closest papers by meaning
- Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models (CVPR 2024 · similarity 0.92)
- Dual Semantic Guidance for Open Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.91)
- Open-Vocabulary Segmentation with Semantic-Assisted Calibration (CVPR 2024 · similarity 0.91)
- Training-free Open-Vocabulary Semantic Segmentation via Diverse Prototype Construction and Sub-region Matching (AAAI 2025 · similarity 0.91)
- Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generation (CVPR 2024 · similarity 0.91)
- Unveiling the Knowledge of CLIP for Training-Free Open-Vocabulary Semantic Segmentation (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
