AI Paper Atlas

Research map / Detection and segmentation / Open-set and few-shot detection

Open-vocabulary Panoptic Segmentation with Embedding Modulation

Xi Chen, Shuang Li, Ser-Nam Lim, Antonio Torralba, Hengshuang Zhao

ICCV 2023 · Detection and segmentation › Open-set and few-shot detection · approach: Open-vocabulary semantic segmentation · 33 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add mask filtering using classification score times IoU score to CLIP-embedding recognition (ablation)CAG-Seg + CLIP Embeds · ADE20K panopticPQ: 4.9 -> 10.7Improved
use query embeddings instead of CLIP embeddings for recognition (ablation)CAG-Seg · COCO panopticPQ: 22.7 -> 51.9Improved
add Embedding Modulation (fusion and logits debiasing) (ablation)OPSNet · ADE20K panopticPQ: 14.6 -> 17.7Improved
add logits debiasing on top of embedding fusion (ablation)OPSNet · ADE20K panopticPQ: 16.9 -> 17.7Improved
naive ensemble with more CLIP weight (ablation)query embeddings · COCO vs ADE20K panopticPQ: COCO 51.4 -> 43.8; ADE20K 16.4 -> 18.1 (peak at 3x)Mixed
mask-pooling with one CLIP pass instead of masking or cropping each object (ablation)CLIP embedding extraction · ADE20K panopticPQ: 9.4 -> 10.7Improved
use class score times IoU for mask filtering (ablation)query embeddings · ADE20K panopticPQ: 13.6 -> 14.5Improved
add cross-attention to gather CLIP features into query embeddings (ablation)CAG-Seg + Query Embeds · ADE20K panopticPQ: 14.5 -> 14.6No clear effect
add ImageNet-Val image-level classification supervision (ablation)OPSNet ResNet-50 · COCO panopticPQ: 52.4 -> 51.3Hurt
add ImageNet classification and mask supervision (decoupled supervision) (ablation)OPSNet ResNet-50 · ADE20K panopticPQ: 17.7 -> 19.0Improved
OPSNet versus Mask2Former on closed-vocabulary COCOMask2Former (Swin-L) · COCO panoptic valPQ: 57.8 -> 57.9Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy