Research map / Image generation / Text-to-image diffusion
MagicSeg: Open-World Segmentation Pretraining via Counterfactural Diffusion-Based Auto-Generation
Kaixin Cai, Pengzhen Ren, Jianhua Han, Yi Zhu, Hang Xu, Jianzhuang Liu, C X Liang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ChatGPT-generated detailed prompts instead of simple 'a photo of class' prompt (ablation) | MagicSeg (40k) · COCO val | mIoU: 28.6 -> 36.0 | Improved |
| add counterfactual contrastive loss with MagicSeg prompts (ablation) | MagicSeg (40k) · COCO val | mIoU: 36.0 -> 37.2 | Improved |
| add counterfactual contrastive loss with simple prompts (ablation) | MagicSeg (40k) · COCO val | mIoU: 28.6 -> 27.5 | Hurt |
| add counterfactual contrastive loss on full dataset (ablation) | MagicSeg (380k) · COCO val | mIoU: 38.1 -> 40.2 | Improved |
| sample 100 categories per image instead of 1 (ablation) | MagicSeg · COCO val | mIoU: 7.7 -> 38.1 | Improved |
| sample 100 categories per image instead of all 1205 (ablation) | MagicSeg · COCO val | mIoU: 31.4 -> 38.1 | Improved |
| train on MagicSeg vs Grounded SAM baseline on LVIS | MagicSeg · LVIS val | mIoU: 0.5 -> 4.9 | Improved |
| train on MagicSeg vs Grounded SAM baseline on PASCAL VOC | MagicSeg · PASCAL VOC val | mIoU: 61.8 -> 62.9 | Improved |
| scale training data from 20 classes (20k) to full 1205 classes (380k) | MagicSeg · COCO val | mIoU: 20.5 -> 40.2 | Improved |
| scale training data from 20 classes (20k) to full 1205 classes (380k) | MagicSeg · PASCAL VOC val | mIoU: 64.3 -> 62.9 | Hurt |
| use MagicSeg synthetic data vs Dataset Diffusion with DeepLabV3 R101 | DeepLabV3 R101 · PASCAL VOC val | mIoU: 64.8 -> 65.3 | Improved |
| use MagicSeg synthetic data vs Dataset Diffusion with Mask2Former R50 | Mask2Former R50 · PASCAL VOC val | mIoU: 60.2 -> 59.3 | Hurt |
Closest papers by meaning
- Seg4Diff: Unveiling Open-Vocabulary Semantic Segmentation in Text-to-Image Diffusion Transformers (NeurIPS 2025 · similarity 0.88)
- Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generation (CVPR 2024 · similarity 0.88)
- Exploring Simple Open-Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.88)
- OpenWorldSAM: Extending SAM2 for Universal Image Segmentation with Language Prompts (NeurIPS 2025 · similarity 0.88)
- Training-free Open-Vocabulary Semantic Segmentation via Diverse Prototype Construction and Sub-region Matching (AAAI 2025 · similarity 0.87)
- APSeg: Auto-Prompt Network for Cross-Domain Few-Shot Semantic Segmentation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
