Research map / Image generation / Fast diffusion sampling
DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Models
Weijia Wu, Yuzhong Zhao, Mike Zheng Shou, Hong Zhou, Chunhua Shen
This paper tried a common change: Add data augmentation
add four data augmentations · VOC 2012 val (Bird, Dog) · mIoU: 84.3 -> 89.5 Improved
What 689 accepted papers report for this change
- 560 improved
- 44 no clear effect
- 26 mixed
- 59 hurt
- Understanding Multimodal Learning: A Loss Landscape Smoothness Perspective ICML 2026 · add data augmentation (jitter, crop, flip) · UPMC-Food101 · accuracy: 92.84 -> 94.93Improved
- AsyncBEV: Cross-modal flow alignment in Asynchronous 3D Object Detection ICLR 2026 · add EMC + asynchronous data augmentation training · nuScenes val, 0 s offset · NDS: 66.7 -> 63.1Hurt
- Towards Understanding Continual Factual Knowledge Acquisition of Language Models: From Theory to Algorithm ICML 2026 · add data augmentation (5 biographies per individual) in pre-training · Biography PT test, unseen templates · EM: 2.71 -> 81.27Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use adaptive threshold instead of fixed threshold 0.5 (ablation) | DiffuMask mask generation · VOC 2012 val (Bird, Dog) | mIoU: 78.9 -> 89.5 | Improved |
| raise fixed binarization threshold from 0.4 to 0.6 (ablation) | Attention-map masks · VOC 2012 val (Bird, Dog) | mIoU: 85.3 -> 44.4 | Hurt |
| add retrieval-based prompt (ablation) | DiffuMask prompts · VOC 2012 val (Bird, Dog) | mIoU: 76.9 -> 77.7 | Improved |
| add 10 sub-classes to retrieval prompt (ablation) | DiffuMask prompts · VOC 2012 val (Bird, Dog) | mIoU: 77.7 -> 87.6 | Improved |
| increase number of sub-classes from 10 to 100 (ablation) | DiffuMask prompts · VOC 2012 val (Bird, Dog) | mIoU: 87.6 -> 89.5 | Improved |
| increase noise-learning prune ratio alpha from 0.3 to 0.7 (ablation) | DiffuMask noise learning · VOC 2012 val (Bird, Dog) | mIoU: 83.2 -> 89.5 | Improved |
| add four data augmentations (ablation) | DiffuMask training · VOC 2012 val (Bird, Dog) | mIoU: 84.3 -> 89.5 | Improved |
| use stronger backbone Swin-B instead of ResNet50 (ablation) | Mask2Former on DiffuMask data · VOC 2012 val | mIoU: 70.3 -> 87.3 (5-class average) | Improved |
| use Swin-L instead of Swin-B (ablation) | Mask2Former on DiffuMask data · VOC 2012 val | mIoU: 87.3 -> 87.4 | No clear effect |
| use pseudo mask from pretrained Mask2Former instead of DiffuMask mask (ablation) | Synthetic images · VOC 2012 val | mIoU: 76.3 -> 82.7 | Improved |
| finetune on 5.0k real images after DiffuMask pretraining | Mask2Former Swin-B · VOC 2012 val | mIoU: 70.6 -> 84.9 | Improved |
Later papers that compared with this method
6 later accepted papers report DiffuMask as a baseline in their results tables; 5 of them report a better number on most of the results they share with it.
- MMDiff: Extending Diffusion Transformers for Multi-Modal Generation (ECCV 2026 · better on 3 of 3 shared results)
- JoDiffusion: Jointly Diffusing Image with Pixel-Level Annotations for Semantic Segmentation Promotion (AAAI 2026 · better on 1 of 1 shared result)
- SeeDiff: Off-the-Shelf Seeded Mask Generation from Diffusion Models (AAAI 2025 · better on 5 of 5 shared results)
- FreeGen: Bridging Visual-Linguistic Discrepancies Towards Diffusion-based Pixel-level Data Synthesis (AAAI 2025 · better on 1 of 1 shared result)
- AlignDiff: Aligning Diffusion Models for General Few-Shot Segmentation (ECCV 2024 · better on 2 of 2 shared results)
- DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Models (NeurIPS 2023 · better on 1 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- $\mathbf{D^{3}S^{2}}$: Diffusion-Guided Dataset Distillation for Semantic Segmentation (NeurIPS 2026 · similarity 0.90)
- Open-Vocabulary Attention Maps with Token Optimization for Semantic Segmentation in Diffusion Models (CVPR 2024 · similarity 0.90)
- Seg4Diff: Unveiling Open-Vocabulary Semantic Segmentation in Text-to-Image Diffusion Transformers (NeurIPS 2025 · similarity 0.89)
- High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity (ICLR 2025 · similarity 0.89)
- SeeDiff: Off-the-Shelf Seeded Mask Generation from Diffusion Models (AAAI 2025 · similarity 0.88)
- EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
