Research map / Image generation / Text-to-image diffusion
Open-Vocabulary Panoptic Segmentation With Text-to-Image Diffusion Models
Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiaolong Wang, Shalini De Mello
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use diffusion features rather than CLIP(H) visual features (ablation) | ODISE · ADE20K panoptic segmentation (512² ablation) | PQ: 21.2 -> 23.3 | Improved |
| Generate an implicit caption instead of using an empty caption (ablation) | ODISE · ADE20K panoptic segmentation | PQ: 21.8 -> 23.3 | Improved |
| Use an implicit captioner instead of a heuristic captioner (ablation) | ODISE · ADE20K panoptic segmentation | PQ: 22.2 -> 23.3 | Improved |
| Use an implicit captioner instead of BLIP (ablation) | ODISE · ADE20K panoptic segmentation | PQ: 22.3 -> 23.3 | Improved |
| Use an implicit captioner instead of BLIP (ablation) | ODISE · COCO panoptic segmentation | PQ: 44.1 -> 44.2 | No clear effect |
| Increase the diffusion time step from 0 to 100 (ablation) | ODISE · ADE20K panoptic segmentation | PQ: 23.3 -> 22.8 | Hurt |
| Increase the diffusion time step from 0 to 200 (ablation) | ODISE · ADE20K panoptic segmentation | PQ: 23.3 -> 21.5 | Hurt |
| Increase the diffusion time step from 0 to 500 (ablation) | ODISE · ADE20K panoptic segmentation | PQ: 23.3 -> 20.9 | Hurt |
| Concatenate features from time steps 0, 100, and 200 rather than using step 0 alone (ablation) | ODISE · ADE20K panoptic segmentation | PQ: 23.3 -> 23.1 | No clear effect |
| Learn the diffusion time step rather than fixing it at 0 (ablation) | ODISE · ADE20K panoptic segmentation | PQ: 23.3 -> 22.8 | Hurt |
| Add discriminative-model predictions to diffusion-model predictions (ablation) | ODISE · ADE20K panoptic segmentation | PQ: 20.1 -> 23.3 | Improved |
| Add discriminative-model predictions to diffusion-model predictions (ablation) | ODISE · COCO panoptic segmentation | PQ: 42.3 -> 44.2 | Improved |
Closest papers by meaning
- Diffusion Models for Open-Vocabulary Segmentation (ECCV 2024 · similarity 0.92)
- Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generation (CVPR 2024 · similarity 0.88)
- USE: Universal Segment Embeddings for Open-Vocabulary Image Segmentation (CVPR 2024 · similarity 0.88)
- Training-free Open-Vocabulary Semantic Segmentation via Diverse Prototype Construction and Sub-region Matching (AAAI 2025 · similarity 0.88)
- OpenWorldSAM: Extending SAM2 for Universal Image Segmentation with Language Prompts (NeurIPS 2025 · similarity 0.88)
- CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
