Research map / Detection and segmentation / Open-set and few-shot detection

A Generative Approach at the Instance-Level for Image Segmentation Under Limited Training Data Conditions (Student Abstract)

AAAI 2025 · Detection and segmentation › Open-set and few-shot detection · approach: Amodal and open-world instance segmentation · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes instance-level synthesis with diffusion-based generators (BlendedDiff-based) to augment limited Cityscapes data for instance segmentation, reporting a ~12% AP gain in initial experiments.

Problem

Instance segmentation requires abundant pixel-level annotations, which are costly; the paper targets settings with limited training data and lacking detailed annotations.

Approach

Conditional instance-level synthesis: an instance-wise diffusion generator produces variants of annotated instances (guided by class prompts and masks) to enrich training data for instance segmentation models; a further idea is embedding a generative model's extractor directly into the segmentation model with a denoising-based decoder and few-shot training.

Key results

• With BlendedDiff-based synthesis, FastInst (R50-FPN) AP rises from 27.65 to 36.52 on Cityscapes validation.
• OneFormer with ConvNext-L backbone improves AP from 21.75 to 38.93 with synthesis; Swin-L improves from 25.68 to 35.75.
• BlendedDiff achieved the best CLIPScore (0.87) and FID (16.28) among GLIGEN, DiffInpainting and BlendedDiff in the initial generation comparison.

Measured effects reported in the paper

Surprising findings

• The Swin-L OneFormer variant's AP gain is reported as a 10.07-point improvement while the paper's text states an average 12.04% improvement; the reported averages are not clearly reconciled with the table.

Limitations

• Only preliminary results on a single dataset (Cityscapes) with no method ablations or significance analysis.
• Generative-extractor embedding and few-shot optimization are proposed but not evaluated.
• Image quality is only indirectly measured via CLIPScore and FID.

Experimental setup

Datasets: Cityscapes

Benchmarks: Cityscapes

Metrics: AP AP50 CLIPScore FID mIoU

Baselines: FastInst OneFormer GLIGEN DiffInpainting

Builds on: BlendedDiff GLIGEN FastInst OneFormer DiffusionDet-style denoising generative segmentation (Gu et al. 2024)

Closest papers by meaning

• Sample-Efficient Multi-Round Generative Data Augmentation for Long-Tail Instance Segmentation (NeurIPS 2025 · similarity 0.89)
• Generative Semantic Segmentation (CVPR 2023 · similarity 0.88)
• Training-Free Dataset Pruning for Instance Segmentation (ICLR 2025 · similarity 0.88)
• DiverGen: Improving Instance Segmentation by Learning Wider Data Distribution with More Diverse Generative Data (CVPR 2024 · similarity 0.87)
• Learning Hierarchical Image Segmentation For Recognition and By Recognition (ICLR 2024 · similarity 0.87)
• Generative Active Learning for Long-tailed Instance Segmentation (ICML 2024 · similarity 0.86)