Research map / Detection and segmentation / Open-set and few-shot detection
OTSeg: Multi-prompt Sinkhorn Attention for Zero-Shot Semantic Segmentation
Kwanyoung Kim, Yujin Oh, Jong Chul Ye
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace cross-attention with MPSA (multi-prompt Sinkhorn attention) plus multiple text prompts (ablation) | OTSeg decoder · VOC 2012 (inductive) | hIoU: 77.6 -> 87.1 | Improved |
| naively add multiple text prompts to standard cross-attention (ablation) | ZegCLIP-style cross-attention decoder · VOC 2012 (inductive) | hIoU: 84.3 -> 77.6 | Hurt |
| naively add multiple text prompts to standard cross-attention (ablation) | ZegCLIP-style cross-attention decoder · PASCAL Context (inductive) | hIoU: 50.3 -> 54.5 | Improved |
| ensemble decoder output with refined score map (OTSeg+) (ablation) | OTSeg · VOC 2012 (inductive) | hIoU: 84.5 -> 87.1 | Improved |
| ensemble decoder output with refined score map (OTSeg+) (ablation) | OTSeg · PASCAL Context (inductive) | hIoU: 54.8 -> 57.7 | Improved |
| ensemble decoder output with refined score map (OTSeg+) (ablation) | OTSeg · COCO-Stuff164K (fully supervised) | hIoU: 50.5 -> 50.1 | Hurt |
| use MPS refined score map only instead of decoder output (ablation) | OTSeg · COCO-Stuff164K (inductive) | hIoU: 41.4 -> 39.3 | Hurt |
| increase number of text prompts from 6 to 8 (ablation) | OTSeg+ · PASCAL Context (inductive) | hIoU: 50.3 -> 57.7 | Improved |
| increase number of text prompts from 6 to 8 (ablation) | OTSeg+ · VOC 2012 (inductive) | hIoU: 87.1 -> 86.1 | Hurt |
| OTSeg over ZegCLIP in cross-dataset transfer from COCO-156 to PASCAL Context | OTSeg (inductive) · COCO-156 to PASCAL Context | mIoU: 41.2 -> 49.3 | Improved |
| OTSeg MPSA overhead relative to ZegCLIP | OTSeg · Efficiency (3090 GPU) | FPS: 25.6 -> 23.6 | Hurt |
Later papers that compared with this method
1 later accepted paper reports OTSeg as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- Object-Centric Refinement for Enhanced Zero-Shot Segmentation (ICLR 2026 · better on 8 of 8 shared results)
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- ZegCLIP — ZegCLIP: Towards Adapting CLIP for Zero-Shot Semantic Segmentation (CVPR 2023) · better on 6 of 7 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- APSeg: Auto-Prompt Network for Cross-Domain Few-Shot Semantic Segmentation (CVPR 2024 · similarity 0.89)
- DPSeg: Dual-Prompt Cost Volume Learning for Open-Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.88)
- Seg4Diff: Unveiling Open-Vocabulary Semantic Segmentation in Text-to-Image Diffusion Transformers (NeurIPS 2025 · similarity 0.87)
- A Simple Framework for Open-Vocabulary Zero-Shot Segmentation (ICLR 2025 · similarity 0.87)
- Spectral Prompt Tuning: Unveiling Unseen Classes for Zero-Shot Semantic Segmentation (AAAI 2024 · similarity 0.87)
- CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
