Research map / Detection and segmentation / Open-set and few-shot detection
A Surprisingly Simple Approach to Generalized Few-Shot Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes base-class mining (BCM), a training-light GFSS method that maps base classes to related novel classes via co-occurrence and trains per-base-class linear heads on frozen features, with a theorem that base-class mIoU is preserved for unmapped base classes.
Problem
Generalized few-shot semantic segmentation must recognize novel classes from few annotated examples while maintaining the performance of a pre-trained base-class model on base classes, without costly meta-learning or transductive inference.
Approach
Count pixel co-occurrences between base-model predictions and novel-class annotations to select, for each novel class, its top-s related base class; then, for each chosen base class, train a linear classifier on frozen base-model features that separates that base class from its mapped novel classes, and overwrite base predictions only for chosen base classes. Base classes not chosen keep their ori
Key results
- In 1-shot PASCAL-5i, BCM reaches 41.24 novel mIoU vs. 35.11 for DIaM (+6.1 points).
- In 1-shot PASCAL-10i, BCM reaches 35.94 novel mIoU vs. 31.29 for DIaM (+4.7 points).
- In 1-shot COCO-20i, BCM reaches 18.28 novel mIoU vs. 17.22 for DIaM (+1.0 points), with training under 1 minute on PASCAL-5i 1-shot.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BCM instead of DIaM | GFSS with ResNet-50 · PASCAL-5i 1-shot | Novel mIoU: 35.11 -> 41.24 | Helped |
| Use BCM instead of DIaM | GFSS with ResNet-50 · PASCAL-10i 1-shot | Novel mIoU: 31.29 -> 35.94 | Helped |
| Use BCM instead of DIaM | GFSS with ResNet-50 · COCO-20i 1-shot | Novel mIoU: 17.22 -> 18.28 | Helped |
| Use BCM instead of DIaM | GFSS with ResNet-50 · PASCAL-5i 5-shot | Novel mIoU: 55.31 -> 55.36 | Helped |
| Use BCM instead of DIaM | GFSS with ResNet-50 · PASCAL-10i 5-shot | Novel mIoU: 51.89 -> 53.49 | Helped |
| Use BCM instead of DIaM | GFSS with ResNet-50 · COCO-20i 5-shot | Novel mIoU: 28.73 -> 30.60 | Helped |
| Add Tukey preprocessing (ablation) | BCM without ensemble learning · PASCAL-5i 1-shot | Novel mIoU: 38.13 -> 41.24 | Helped |
| Add Tukey preprocessing (ablation) | BCM without ensemble learning · PASCAL-10i 1-shot | Novel mIoU: 34.56 -> 35.94 | Helped |
| Add Tukey preprocessing (ablation) | BCM without ensemble learning · COCO-20i 1-shot | Novel mIoU: 18.03 -> 18.28 | Helped |
| Add ensemble learning (ablation) | BCM without preprocessing · PASCAL-5i 5-shot | Novel mIoU: 49.83 -> 53.53 | Helped |
| Add ensemble learning (ablation) | BCM without preprocessing · PASCAL-10i 5-shot | Novel mIoU: 48.80 -> 52.27 | Helped |
| Add ensemble learning (ablation) | BCM without preprocessing · COCO-20i 5-shot | Novel mIoU: 26.73 -> 30.48 | Helped |
Surprising findings
- Most novel classes are mapped to the background or a single similar base class, and typically only a few base classes (at most four in the 5-shot COCO-20i setting) need new heads.
- Increasing top-s beyond 1 tended to degrade base and novel performance rather than help.
Limitations
- Improvement in the 5-shot setting is slight.
- Base-class performance is maintained but not improved using novel-class data.
- Inference slows somewhat because linear heads are switched on CPU via scikit-learn.
Experimental setup
Datasets: PASCAL-5i PASCAL-10i COCO-20i
Benchmarks: PASCAL-5i PASCAL-10i COCO-20i
Metrics: mIoU Base mIoU Novel mIoU Mean mIoU
Baselines: CAPL BAM DIaM
Builds on: DIaM CAPL BAM PSPNet
Closest papers by meaning
- Enhancing Generalized Few-Shot Semantic Segmentation via Effective Knowledge Transfer (AAAI 2025 · similarity 0.94)
- Unlocking the Potential of Pre-trained Vision Transformers for Few-Shot Semantic Segmentation through Relationship Descriptors (CVPR 2024 · similarity 0.91)
- Label-Efficient Few-Shot Semantic Segmentation with Unsupervised Meta-Training (AAAI 2024 · similarity 0.90)
- DSV-LFS: Unifying LLM-Driven Semantic Cues with Visual Features for Robust Few-Shot Segmentation (CVPR 2025 · similarity 0.88)
- Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language Model (CVPR 2025 · similarity 0.88)
- CGMGM: A Cross-Gaussian Mixture Generative Model for Few-Shot Semantic Segmentation (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.