Research map / Detection and segmentation / Open-set and few-shot detection
Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| align caption object words to teacher-selected regions and use them for cross-modal pseudo-labeling | XPM · MS-COCO instance segmentation | target mAP: +3.1 | Helped |
| align caption object words to teacher-selected regions and use them for cross-modal pseudo-labeling | XPM · Open Images & Conceptual Captions instance segmentation | target mAP: +5.7 | Helped |
| add the cross-modal loss from caption-grounded object regions (ablation) | XPM student · MS-COCO and Open Images & Conceptual Captions | segmentation performance | Helped |
| add pseudo-mask supervision through the mask loss (ablation) | XPM student · MS-COCO | target-class segmentation performance | Helped |
| add pseudo-mask supervision through the mask loss (ablation) | XPM student · Conceptual Captions | target-class segmentation performance | No clear effect |
| reweight cross-modal loss using estimated pseudo-mask noise (ablation) | XPM student · caption datasets | segmentation performance | Helped |
| add pseudo-mask noise estimation and reweighting to both cross-modal and mask losses (ablation) | XPM Robust Student · Open Images & Conceptual Captions constrained | target mAP: 30.2 -> 31.6 | Helped |
| use pixel-wise stochastic noise to regulate mask loss only (ablation) | XPM · Open Images & Conceptual Captions constrained | target mAP: 30.2 -> 29.8 | Hurt |
| reweight cross-modal loss using class prediction confidence (ablation) | XPM · Open Images & Conceptual Captions constrained | target mAP: 30.2 -> 28.4 | Hurt |
| reweight cross-modal loss using aggregated pixel confidence (ablation) | XPM · Open Images & Conceptual Captions constrained | target mAP: 30.2 -> 30.1 | No clear effect |
| reweight cross-modal loss using prediction entropy from dropout passes (ablation) | XPM · Open Images & Conceptual Captions constrained | target mAP: 30.2 -> 29.7 | Hurt |
| use cross-modal pseudo-labeling with caption semantics rather than conventional visual-only pseudo-labeling | XPM · MS-COCO generalized instance segmentation | target mAP: 17.1 -> 21.6 | Helped |
Closest papers by meaning
- Mask-Adapter: The Devil is in the Masks for Open-Vocabulary Segmentation (CVPR 2025 · similarity 0.89)
- CLIP as RNN: Segment Countless Visual Concepts without Training Endeavor (CVPR 2024 · similarity 0.89)
- Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models (CVPR 2024 · similarity 0.89)
- Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generation (CVPR 2024 · similarity 0.88)
- Training-free Open-Vocabulary Semantic Segmentation via Diverse Prototype Construction and Sub-region Matching (AAAI 2025 · similarity 0.88)
- Effective SAM Combination for Open-Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.