Research map / Detection and segmentation / Open-set and few-shot detection
Open-vocabulary Object Segmentation with Diffusion Models
Ziyi Li, Qinye Zhou, Xiaoyun Zhang, Ya Zhang, Yanfeng Wang, Weidi Xie
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| ignore all-zero detector masks when training the grounding module (ablation) | grounding module on Stable Diffusion · PASCAL-sim one object, unseen | mIoU: 71.18 -> 83.19 | Improved |
| ignore all-zero masks (two-object unseen) (ablation) | grounding module · PASCAL-sim two objects, unseen | mIoU: 44.22 -> 57.93 | Improved |
| add more seen categories to training (ablation) | grounding module · COCO-sim | mIoU: unseen (one object) 72.42 -> 76.81; seen 94.81 -> 83.85 | Mixed |
| train on a mixture of one- and two-object images instead of only one-object images (ablation) | grounding module · PASCAL-sim two objects | mIoU: single: seen 43.89 -> mixture 78.93 | Improved |
| use synthetic training data instead of real PASCAL VOC data (ablation) | grounding module · PASCAL-sim one object, unseen | mIoU: 61.26 -> 83.19 | Improved |
| scale synthetic training set from 10k to 40k images (ablation) | grounding module · PASCAL-sim one object, unseen | mIoU: 70.04 -> 83.19 | Improved |
| extract diffusion features at t=5 rather than later timesteps | grounding module · PASCAL-sim | mIoU | Improved |
| combine synthetic and real data when training segmenter (ablation) | MaskFormer segmenter · PASCAL VOC zero-shot segmentation | unseen mIoU: 63.2 -> 71.3 | Improved |
| construct synthetic dataset with more complicated prompts (ablation) | MaskFormer segmenter · PASCAL VOC zero-shot segmentation | unseen mIoU: 71.3 -> 74.4 | Improved |
| train segmenter on mixture of one- and two-object synthetic images (ablation) | MaskFormer segmenter · PASCAL VOC zero-shot segmentation | unseen mIoU: 50.0 / 60.1 -> 63.2 | Improved |
| add trained grounding module versus DAAM attention maps | Stable Diffusion · PASCAL-sim | mIoU: 62.90 -> 83.08 (one object unseen) | Improved |
Closest papers by meaning
- Open-Vocabulary Attention Maps with Token Optimization for Semantic Segmentation in Diffusion Models (CVPR 2024 · similarity 0.91)
- Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generation (CVPR 2024 · similarity 0.91)
- Seg4Diff: Unveiling Open-Vocabulary Semantic Segmentation in Text-to-Image Diffusion Transformers (NeurIPS 2025 · similarity 0.90)
- Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation (NeurIPS 2024 · similarity 0.90)
- VidSeg: Training-free Video Semantic Segmentation based on Diffusion Models (CVPR 2025 · similarity 0.89)
- Segment Anyword: Mask Prompt Inversion for Open-Set Grounded Segmentation (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
