Research map / Detection and segmentation / Open-set and few-shot detection
Open Vocabulary Semantic Segmentation With Patch Aligned Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace CLIP's CLS-to-CLS compatibility with PACL patch-weighted compatibility | CLIP ViT-B/16 · Pascal VOC | patch classification accuracy: 52.49 -> 96.51 | Helped |
| Replace CLIP's CLS-to-CLS compatibility with PACL patch-weighted compatibility | CLIP ViT-L/14 · Pascal VOC | patch classification accuracy: 27.91 -> 95.33 | Helped |
| Train CLIP with PACL rather than use vanilla CLIP patch-text similarity | Zero-shot segmentation · Pascal VOC, Pascal Context, COCO Stuff, ADE20K | mIoU: 8.4 -> 72.3; 2.3 -> 50.1; 2.6 -> 38.8; 1.3 -> 31.4 | Helped |
| Use PACL with CLIP ViT-B/16 rather than the strongest listed prior method | Zero-shot segmentation · Pascal VOC, Pascal Context, COCO Stuff, ADE20K | mIoU: 72.2 -> 72.3; 48.2 -> 50.1; 24.3 -> 38.8; 28.6 -> 31.4 | Helped |
| Add YFCC15M to GCC12M training data (ablation) | PACL with CLIP B/16 · Pascal VOC | mIoU: 64.1 -> 69.2 | Helped |
| Add GCC3M to GCC12M + YFCC15M training data (ablation) | PACL with CLIP B/16 · Pascal VOC | mIoU: 69.2 -> 72.3 | Helped |
| Use CLIP L/14 instead of CLIP B/16 as vision and text encoders (ablation) | PACL trained on GCC12M · Pascal VOC | mIoU: 64.1 -> 62.7 | Hurt |
| Use DINO B/16 vision encoder with CLIP B/16 text encoder instead of CLIP B/16 vision and text encoders (ablation) | PACL trained on GCC12M · Pascal VOC | mIoU: 64.1 -> 55.4 | Hurt |
| Use CLIP L/14 instead of CLIP B/16 as vision and text encoders (ablation) | PACL trained on GCC12M + YFCC15M · Pascal VOC | mIoU: 69.2 -> 68.4 | Hurt |
| Use DINO B/16 vision encoder with CLIP B/16 text encoder instead of CLIP B/16 vision and text encoders (ablation) | PACL trained on GCC12M + YFCC15M · Pascal VOC | mIoU: 69.2 -> 62.6 | Hurt |
| Use CLIP L/14 instead of CLIP B/16 as vision and text encoders (ablation) | PACL trained on GCC3M + GCC12M + YFCC15M · Pascal VOC | mIoU: 72.3 -> 71.7 | Hurt |
| Use DINO B/16 vision encoder with CLIP B/16 text encoder instead of CLIP B/16 vision and text encoders (ablation) | PACL trained on GCC3M + GCC12M + YFCC15M · Pascal VOC | mIoU: 72.3 -> 64.8 | Hurt |
Closest papers by meaning
- SmartCLIP: Modular Vision-language Alignment with Identification Guarantees (CVPR 2025 · similarity 0.86)
- Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models (CVPR 2024 · similarity 0.86)
- A Simple Framework for Open-Vocabulary Zero-Shot Segmentation (ICLR 2025 · similarity 0.86)
- Toward Modality Gap: Vision Prototype Learning for Weakly-supervised Semantic Segmentation with CLIP (AAAI 2025 · similarity 0.86)
- Seeing What Matters: Empowering CLIP with Patch Generation-to-Selection (CVPR 2025 · similarity 0.86)
- TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP without Training (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.