Research map / Detection and segmentation / Open-set and few-shot detection
Open-Vocabulary Universal Image Segmentation with MaskCLIP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MaskCLIP instead of OpenSeg | open-vocabulary semantic segmentation · ADE20K A-150 | mIoU: 21.1 -> 23.7 | Helped |
| Use MaskCLIP instead of SimSeg | open-vocabulary semantic segmentation · PASCAL Context P-59 | mIoU: 47.7 -> 45.9 | Hurt |
| Add Relative Mask Attention to Mask Class Tokens | MaskCLIP semantic segmentation · ADE20K A-150 | mIoU: 14.9 -> 23.7 | Helped |
| Add Relative Mask Attention | open-vocabulary panoptic segmentation · ADE20K val | PQ: 9.565 -> 15.121 | Helped |
| Use MaskCLIP rather than MaskCLIP w/o RMA | open-vocabulary instance segmentation · cross-dataset ADE20K | AP: 4.263 -> 5.989 | Helped |
| Use MaskCLIP rather than MaskCLIP w/o RMA | open-vocabulary instance segmentation · cross-dataset LVIS | AP: 5.762 -> 8.404 | Helped |
| Use MaskCLIP rather than XPM | open-vocabulary instance segmentation · COCO split generalized | All AP: 36.3 -> 37.2 | Helped |
| Extract multiple mask features with Mask Class Tokens rather than repeated CLIP image passes | feature extraction for 100 masks · single image, single 3090 GPU | speed: ~3s -> ~0.6s; ~4x | Helped |
| Use ground-truth masks rather than predicted proposals (ablation) | MaskCLIP panoptic segmentation · ADE20K val | PQ: 15.1 -> 35.8 | Helped |
| Use ground-truth masks rather than predicted proposals (ablation) | MaskCLIP semantic segmentation · ADE20K val | mIoU: 23.7 -> 31.7 | Helped |
| Add mask refinement (ablation) | MaskCLIP panoptic segmentation · ADE20K val | SQ: 66.361 -> 70.479 | Helped |
| Place RMA in layers 6, 12, 18, 24 instead of only the last four layers (ablation) | MaskCLIP panoptic segmentation · ADE20K val | PQ: 14.673 -> 15.121 | Helped |
Closest papers by meaning
- Effective SAM Combination for Open-Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.90)
- Towards Open-Vocabulary Semantic Segmentation Without Semantic Labels (NeurIPS 2024 · similarity 0.90)
- Mask-Adapter: The Devil is in the Masks for Open-Vocabulary Segmentation (CVPR 2025 · similarity 0.90)
- USE: Universal Segment Embeddings for Open-Vocabulary Image Segmentation (CVPR 2024 · similarity 0.89)
- CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor (CVPR 2024 · similarity 0.88)
- Open-Vocabulary Semantic Segmentation with Image Embedding Balancing (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.