AI Paper Atlas

Research map / Detection and segmentation / Open-set and few-shot detection

Open-vocabulary Object Segmentation with Diffusion Models

Ziyi Li, Qinye Zhou, Xiaoyun Zhang, Ya Zhang, Yanfeng Wang, Weidi Xie

ICCV 2023 · Detection and segmentation › Open-set and few-shot detection · approach: Open-vocabulary semantic segmentation · 89 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
ignore all-zero detector masks when training the grounding module (ablation)grounding module on Stable Diffusion · PASCAL-sim one object, unseenmIoU: 71.18 -> 83.19Improved
ignore all-zero masks (two-object unseen) (ablation)grounding module · PASCAL-sim two objects, unseenmIoU: 44.22 -> 57.93Improved
add more seen categories to training (ablation)grounding module · COCO-simmIoU: unseen (one object) 72.42 -> 76.81; seen 94.81 -> 83.85Mixed
train on a mixture of one- and two-object images instead of only one-object images (ablation)grounding module · PASCAL-sim two objectsmIoU: single: seen 43.89 -> mixture 78.93Improved
use synthetic training data instead of real PASCAL VOC data (ablation)grounding module · PASCAL-sim one object, unseenmIoU: 61.26 -> 83.19Improved
scale synthetic training set from 10k to 40k images (ablation)grounding module · PASCAL-sim one object, unseenmIoU: 70.04 -> 83.19Improved
extract diffusion features at t=5 rather than later timestepsgrounding module · PASCAL-simmIoUImproved
combine synthetic and real data when training segmenter (ablation)MaskFormer segmenter · PASCAL VOC zero-shot segmentationunseen mIoU: 63.2 -> 71.3Improved
construct synthetic dataset with more complicated prompts (ablation)MaskFormer segmenter · PASCAL VOC zero-shot segmentationunseen mIoU: 71.3 -> 74.4Improved
train segmenter on mixture of one- and two-object synthetic images (ablation)MaskFormer segmenter · PASCAL VOC zero-shot segmentationunseen mIoU: 50.0 / 60.1 -> 63.2Improved
add trained grounding module versus DAAM attention mapsStable Diffusion · PASCAL-simmIoU: 62.90 -> 83.08 (one object unseen)Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy